WO2021197035A1 - Method and device for jointly training service prediction model by two parties for protecting data privacy - Google Patents

Abstract

Embodiments of the present invention provide a method and device for jointly training a service prediction model by two parties for protecting data privacy. The two parties each has a part of feature data. In a model iteration process, the two parties obtain encrypted fragments of a product result of a total feature matrix X and a total parameter matrix W by means of security matrix multiplication; the two encrypted fragments are summarized by a second party having a label to obtain an encrypted product result Z; the second party obtains an encrypted error E on the basis of the product result Z and an encrypted label Y, and carries out secret sharing on the encrypted error E under homomorphic encryption; thus the two parties respectively obtain error fragments; then the two parties obtain corresponding gradient fragments by means of secret sharing and security matrix multiplication on the basis of the error fragments and respective feature matrixes; then a first party updates, by utilizing the gradient fragments of the first party, the parameter fragments maintained by the first party, and the second party updates, by utilizing the gradient fragments of the second party, the parameter fragments maintained by the second party. Therefore, secure joint training for protecting data privacy is realized.

Description

Method and device for joint training of business prediction model by two parties that protect data privacy Technical field

One or more embodiments of this specification relate to the fields of data security and machine learning, and in particular, to methods and devices for joint training of business prediction models by both parties.

Background technique

The data needed for machine learning often involves multiple fields. For example, in a business classification analysis scenario based on machine learning, the electronic payment platform owns the merchant's transaction flow data, the e-commerce platform stores the merchant's sales data, and the banking institution owns the merchant's loan data. Data often exists in the form of islands. Due to industry competition, data security, user privacy and other issues, data integration is facing great resistance. It is difficult to integrate data scattered on various platforms to train machine learning models. Under the premise of ensuring that data is not leaked, the use of multi-party data to jointly train machine learning models has become a major challenge at present.

Commonly used machine learning models include logistic regression models, linear regression models, and neural network models, among which logistic regression models can effectively perform tasks such as sample classification and prediction, linear regression models can effectively predict the regression values of samples, neural network models Various prediction tasks can be performed through the combination of multiple layers of neurons. In the training process of the above models, the process of using the calculation between the feature data and the model parameter data to obtain the prediction result, and determining the gradient according to the prediction result, and then adjusting the model parameters. In the case of multiple parties jointly training a machine learning model, how to coordinate the operations of the above-mentioned various stages without revealing the private data of all parties, including feature data and model parameter data, is a practical problem to be solved.

Therefore, it is hoped to provide an improved solution to ensure that the private data of each party is not leaked and ensure data security when the two parties jointly train the business prediction model.

Summary of the invention

One or more embodiments of this specification describe a method and device for jointly training a business prediction model by both parties, in which parameter fragmentation in the iterative process is used to ensure that data privacy is not leaked, and to ensure the security of private data in joint training.

According to a first aspect, there is provided a method for two parties to jointly train a business prediction model to protect data privacy. The two parties include a first party and a second party, and the first party stores first characteristic parts of a plurality of business objects. A first feature matrix X _A formed by the second party; the second party stores a second feature matrix X _B formed by the second feature parts of the multiple business objects, and a label vector Y formed by label values; the method is applied to The second party, the method includes performing model parameter update multiple iterations, wherein each iteration includes:

Based on the locally maintained first parameter second segment and the second parameter second segment, the second encrypted multiplication integral piece of homomorphic encryption is calculated through local matrix multiplication and secure matrix multiplication with the first party, and receiving from the first party a first encrypted by integral piece; wherein a first slice is the second parameter of the second slice for a first characteristic portion of the first process parameter W _a portion; a second parameter The second fragment is a second fragment used to process the second parameter part W _B of the second characteristic part;

A first integrator by the first encryption and the second encryption sheet by sheet integral homomorphic summed and the result is encrypted product Z, which corresponds to the first feature matrix X _A W _A and the first parameter section multiplied Product, and the encrypted value of the sum of the second product of the second feature matrix X _B and the second parameter part W _B;

Perform a homomorphic operation based on the encrypted product result Z and the encrypted value of the label vector Y to obtain an encrypted error vector E, and secretly share the encrypted error vector E to obtain a second error fragment;

Perform matrix multiplication under homomorphic operation on the encryption error vector E and the second characteristic matrix X _B to obtain the second encryption gradient, and perform secret sharing on the second encryption gradient to obtain the second gradient second slice;

Use the second error fragment to _{perform a security matrix multiplication with the first feature matrix X A} in the first party to obtain the second fragment of the second part of the first gradient;

According to the second segment of the second gradient, update the second segment of the second parameter; update the second segment of the first parameter according to the second segment of the second part of the first gradient.

According to one embodiment, before performing multiple iterations to perform model parameter update, the method further includes: initializing the second parameter part W _B , and splitting it into the second parameter first segment and the second parameter second segment through secret sharing. receiving a first parameter from a first side portion of the first parameter W _a secret sharing; fragment, retaining the second parameter of the second fragment, transmits the first fragment of the second parameter to the first party The second fragment.

In an embodiment, after performing the multiple iterations to perform the model parameter update, the method further includes: sending the second segment of the first parameter updated in the last iteration to the first party, and receiving the update from all The first party receives the updated first segment of the second parameter; the second segment of the second parameter updated in the last iteration is combined with the received first segment of the second parameter to obtain the service _{The second parameter part W B} after the prediction model is trained.

In different embodiments, the business object may include one of the following: users, merchants, commodities, and events; the business prediction model is used to predict the classification or regression value of the business object.

According to an embodiment, the service prediction model is a linear regression model; in this case, the homomorphic difference between the encrypted product result Z and the label vector Y can be calculated as the encrypted error vector E.

According to another embodiment, the service prediction model is a logistic regression model; in this case, the encrypted prediction result can be obtained based on the encrypted product result Z according to the Taylor expansion form of the sigmoid function, and the encrypted prediction result and the The encrypted value of the label vector Y is subjected to a homomorphic difference operation to obtain the encrypted error vector E.

Further, in an example summary, before obtaining the encryption error vector E, it also includes calculating the encrypted multi-order product at least according to the first multiplier integral piece and the second multiplier integral piece; in this way, the sigmoid function can be calculated according to In a multi-order Taylor expansion form, an encrypted prediction result is obtained based on the encrypted product result Z and the encrypted multi-factor product, and a homomorphic difference operation is performed on the encrypted prediction result and the encrypted value of the label vector Y to obtain the encrypted error Vector E.

In a specific embodiment, the second multiplicative integral piece of homomorphic encryption is calculated by the following method: the second piece of the first parameter is used to perform the security matrix _{with the first characteristic matrix X A in the first party} Multiply to obtain the second segment of the second processing result of the first feature; locally calculate the product of the second feature matrix X _B and the second segment of the second parameter to obtain the first processing result of the second feature; use the second feature The matrix X _B is multiplied by a security matrix with the first segment of the second parameter in the first party to obtain the second segment of the second processing result of the second feature; the second segment of the second processing result of the first feature is Two shards, the first processing result of the second characteristic, the second shard of the second processing result of the second characteristic are added, and the sum result is homomorphically encrypted with the public key of the first party, Obtain the second encrypted multiplication integral piece.

In an embodiment, the second segment of the second parameter is updated in the following manner, that is, the second parameter is updated by subtracting the product of the second segment of the second gradient and a preset step size. Fragmentation.

According to a second aspect, there is provided a method for two parties to jointly train a business prediction model to protect data privacy. The method is applied to the aforementioned first party. The method includes: performing model parameter update multiple iterations, wherein each iteration includes:

Based on the locally maintained first parameter first slice and the second parameter first slice, the homomorphic encrypted first encrypted multiplication integral slice is calculated through the local matrix multiplication operation and the secure matrix multiplication operation with the second party ; Wherein, the first segment of the first parameter is used to process the first segment of the first parameter part W _A of the first characteristic part; the first segment of the second parameter is used to process the second The first segment of the second parameter part W _{B of the characteristic part;}

Send the first encrypted multiplication integral piece to the second party, so that the second party homomorphically sums the first encrypted multiplication integral piece and the second encrypted multiplication integral piece calculated to obtain the encrypted product result Z, which corresponds to in the first feature with the first parameter matrix X _a W _a portion of the first product of multiplication, and the second feature matrix X _B and the encrypted value and a second product portion of the second parameter multiplied W _B;

Receiving, from the second party, the first error fragment secretly shared with the encryption error vector E, where the encryption error vector E is determined based on the homomorphic operation of the encryption product result Z and the encrypted value of the label vector Y;

Performing a local multiplication operation on the transposition of the first error _{segment and the first feature matrix X A} to obtain the first part of the first gradient;

Use the first feature matrix X _{A to} perform a security matrix multiplication with the second error segment retained in the second party to obtain the first segment of the second part of the first gradient;

Receiving, from the second party, the second gradient first segment that is secretly shared with the second encryption gradient;

According to the first slice of the first part of the first gradient and the first slice of the second part of the first gradient, update the first slice of the first parameter; according to the first slice of the second gradient, update the first slice of the second parameter.

According to an embodiment, before performing multiple iterations to perform the model parameter update, the method further includes: initializing the first parameter part W _A , and splitting it into the first parameter first segment and the first parameter part through secret sharing. Two fragments, the first fragment of the first parameter is reserved, and the second fragment of the first parameter is sent to the second party; the second parameter part W _B secretly shared from the second party is received The first segment of the parameter.

According to an embodiment, after performing the model parameter update multiple iterations, the method further includes: sending the first segment of the second parameter updated in the last iteration to the second party, and receiving the update from the second party. The party receives the updated first parameter second segment; the updated first parameter first segment in the last iteration is combined with the received first parameter second segment to obtain the service prediction model training After the first parameter part W _A.

In a specific embodiment, the first multiplicative integral slice of homomorphic encryption is calculated by the following method: _{the product of the first characteristic matrix X A} and the first slice of the first parameter is calculated locally to obtain the first processing result of the first characteristic; Use the first feature matrix X _{A to} perform security matrix multiplication with the first parameter second segment in the second party to obtain the first segment of the first feature second processing result; use the second parameter The first segment is multiplied by a security matrix with the second feature matrix X _B in the second party to obtain the first segment of the second processing result of the second feature; the first processing result of the first feature is obtained, so The first fragment of the second processing result of the first feature, the first fragment of the second processing result of the second feature is added, and the sum result is homomorphically encrypted with the public key of the first party, Obtain the first encrypted multiplication integral piece.

According to an embodiment, the first segment of the first parameter is updated in the following manner: the product of the sum of the first segment of the first part of the first gradient and the first segment of the second part of the first gradient and a preset step is taken as The adjustment amount is to update the first segment of the first parameter by subtracting the adjustment amount.

According to a third aspect, there is provided a device for two parties to jointly train a business prediction model to protect data privacy. The two parties include a first party and a second party, and the first party stores first characteristic parts of a plurality of business objects. A first feature matrix X _A formed by the second party; the second party stores a second feature matrix X _B formed by the second feature parts of the multiple business objects, and a label vector Y formed by label values; the device is deployed in The second party, the device includes an iterative unit for performing model parameter update multiple times, which further includes:

The multiplication-integral piece calculation unit is configured to calculate the homomorphic encryption based on the locally maintained first parameter second piece and the second parameter second piece through local matrix multiplication and secure matrix multiplication with the first party second encryption by integral sheet, and receiving encrypted by a first integral sheet from the first party; wherein the second fragment is the first parameter for processing the first feature of the first portion of the parameter W _a portion of The second fragment; the second parameter second fragment is the second fragment used to process the second parameter part W _B of the second characteristic part;

The product result determining unit is configured to perform a homomorphic summation on the first encrypted multiplying integral piece and the second encrypted multiplying integral piece to obtain an encrypted product result Z, which corresponds to the first characteristic matrix X _A and the first parameter part The encrypted value of the sum of the first product of W _A and the second product of the second feature matrix X _B and the second parameter part W _B;

The error vector determining unit is configured to perform a homomorphic operation based on the encrypted product result Z and the encrypted value of the label vector Y to obtain an encrypted error vector E, and secretly share the encrypted error vector E to obtain a second error fragment ；

The first gradient determining unit is configured to perform matrix multiplication under the homomorphic operation on the encryption error vector E and the second characteristic matrix X _B to obtain a second encryption gradient, and perform secret sharing of the second encryption gradient to obtain a second encryption gradient. Gradient second slice;

The second gradient determining unit is configured to use the second error _{segment to perform a security matrix multiplication with the first feature matrix X A} in the first party to obtain a second segment of the second part of the first gradient;

The parameter update unit is configured to update the second parameter second slice according to the second slice of the second gradient; update the first parameter second slice according to the second slice of the second part of the first gradient Two slices.

According to a fourth aspect, there is provided an apparatus for both parties to jointly train a service prediction model to protect data privacy, which is deployed in the aforementioned first party. The apparatus includes: an iterative unit for performing model parameter update multiple iterations, which further includes :

The multiply-integral piece calculation unit is configured to calculate the homomorphism based on the first piece of the first parameter and the first piece of the second parameter maintained locally, through the local matrix multiplication operation and the safe matrix multiplication operation with the second party multiply encrypting the encrypted first integral sheet; wherein the first parameter of the first fragment is a first fragment for a first characteristic portion of the first process parameter W _a portion; a second parameter of the first fragment Is the first segment used to process the second parameter part W _B of the second characteristic part;

The multiplying integral piece sending unit is configured to send the first encrypted multiplying integral piece to the second party, so that the second party performs a homomorphic summation of the first encrypted multiplying integral piece and the second encrypted multiplying integral piece calculated by the first encrypted multiplying integral piece to encryption obtained multiplication result Z, which corresponds to a first product of the first feature matrix X _a W _a portion of the first parameter multiplied, and a second product of the second feature matrix X _B W _B with the second parameter section multiplied The encrypted value of the sum;

The error fragment receiving unit is configured to receive the first error fragment secretly shared with the encrypted error vector E from the second party, wherein the encrypted error vector E is based on a homomorphic operation of the encrypted product result Z and the encrypted value of the label vector Y Sure;

A first gradient determining unit, configured to _{perform a local multiplication operation on the transposition of the first error segment and the first feature matrix X A} to obtain the first part of the first gradient;

The second gradient determining unit is configured to use the first feature matrix X _{A to} perform a security matrix multiplication with the second error segment retained in the second party to obtain the first segment of the second part of the first gradient;

The third gradient determining unit is configured to receive, from the second party, the second gradient first fragment that is secretly shared with the second encrypted gradient;

The parameter update unit is configured to update the first segment of the first parameter according to the first segment of the first part of the first gradient and the first segment of the second part of the first gradient; update the second segment of the first parameter according to the first segment of the second gradient The first segment of the parameter.

According to a fifth aspect, there is provided a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed in a computer, the computer is caused to execute the method of the first aspect or the second aspect.

According to a sixth aspect, there is provided a computing device, including a memory and a processor, characterized in that executable code is stored in the memory, and when the processor executes the executable code, the first aspect or the first aspect is implemented. Two-sided approach.

According to the method and device provided in the embodiments of this specification, the two parties participating in the joint training each have a part of characteristic data. In the iterative process of joint training, the two parties not only do not exchange the plaintext of feature data, but also split the model parameter part into parameter shards, and each only maintains the iterative update of the sharding parameters. The model will not be reconstructed until the end of the iteration. parameter. In the iterative process, all parties only maintain parameter shards and exchange some sharding results, and it is almost impossible to infer useful information about private data based on these sharding results. This greatly enhances the privacy data in the joint training process. safety.

Description of the drawings

In order to explain the technical solutions of the embodiments of the present invention more clearly, the following will briefly introduce the drawings used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. A person of ordinary skill in the art can obtain other drawings based on these drawings without creative work.

Fig. 1 is a schematic diagram of an implementation scenario of an embodiment disclosed in this specification;

Figure 2 shows a secret sharing scheme under homomorphic encryption in one embodiment;

Figure 3 shows an implementation scheme of secure matrix multiplication in one embodiment;

Fig. 4 shows a schematic diagram of a process of joint training of a linear regression model by two parties according to an embodiment;

Figure 5 shows part of the implementation process of the first sub-phase in an embodiment;

Fig. 6 shows a schematic diagram of a process of joint training of a logistic regression model between two parties according to an embodiment;

Fig. 7 shows a schematic block diagram of a joint training device deployed in a second party according to an embodiment;

Fig. 8 shows a schematic block diagram of a joint training device deployed in a first party according to an embodiment.

Detailed ways

The following describes the solutions provided in this specification with reference to the accompanying drawings.

As mentioned above, the training process of a typical machine learning model includes a process of obtaining a prediction result from the calculation between feature data and model parameter data, determining the gradient according to the prediction result, and then adjusting the model parameters according to the gradient.

Specifically, assuming that the training data set used to train the machine learning model has n samples, the sample feature of each sample is expressed as x (x can be a vector), and the label is expressed as y, then the training data set can be expressed as:

Through the calculation of the sample feature x of each sample and the model parameter w, the predicted value of the sample can be obtained

If the machine learning model is a linear regression model, the predicted value can be expressed as:

If the machine learning model is a logistic regression model, the predicted value can be expressed as:

In the case of using maximum likelihood probability and stochastic gradient descent, the obtained gradient can be expressed as:

in,

Is the predicted value, y is the label value, the superscript T is the transposition, and x is the feature; therefore, the parameter w can be updated according to the gradient to achieve model training.

As can be seen from the above process, the training process includes several core operations: calculate the product xw of the sample feature x and the model parameter w, and the product xw is used to determine the predicted value

pass through

Obtain the prediction error E; then according to the product of the prediction error E and x, the gradient is obtained.

In the case of a single-party independent training model, the above-mentioned calculations can be easily performed. However, in the case of multi-party joint training of machine learning models, the characteristics of the same sample may be distributed among different participants. Each participant maintains some of the parameters of the model. How to implement the above items without revealing the plaintext data of all parties Computation is the core challenge for realizing data privacy protection in joint training.

In response to the above problems, the inventor proposed that in the scenario where the two parties jointly train the machine learning model, the model parameters of each party should be disassembled into secure parameter fragments. With the help of secret sharing, homomorphic encryption and secure matrix multiplication, the above The operation is also correspondingly disassembled into a safe and secret sharding operation. Through the interaction and joint calculation of the results of the sharding operation by both parties, the above-mentioned operations are realized, thereby realizing safe collaborative training.

Figure 1 is a schematic diagram of an implementation scenario of an embodiment disclosed in this specification. As shown in Figure 1, the scenario of joint training between the two parties involves participant A and participant B, or called the first party and the second party. Each participant can be implemented as any device, platform, server or device cluster with computing and processing capabilities. Both parties must jointly train a business prediction model while protecting data privacy.

The first party A stores part of the features of n business objects in the training sample set, which is called the first feature part. Assuming that the first feature part of each business object is a d1-dimensional vector, then the first feature parts of n business objects constitute an n*d1-dimensional first feature matrix X _A. The second party B stores the second characteristic parts of the n business objects. Assuming that the second feature part of each business object is a d2-dimensional vector, then the second feature parts of n business objects constitute an n*d2-dimensional second feature matrix X _B. It is assumed that the label values of n business objects are also stored in the second party, and the n label values constitute a label vector Y.

For example, in an exemplary scenario, the above-mentioned first party A and second party B are electronic payment platforms and banking institutions, and the two parties need to jointly train a business prediction model to evaluate the user's credit rating. At this point, the business object is the user. Both parties can maintain part of the user's characteristic data. For example, the electronic payment platform maintains the user's electronic payment and transfer related characteristics, which constitutes the above-mentioned first characteristic matrix; the banking institution maintains the user's credit record related characteristics, which constitutes the above-mentioned second Feature matrix. In addition, the banking institution also has a label Y for the user's credit rating.

In another example, the above-mentioned first party A and second party B are an e-commerce platform and an electronic payment platform, and both parties need to jointly train a business prediction model to assess the merchant's fraud risk. At this time, the business object is the merchant. Both parties can maintain part of the characteristic data of the merchants respectively. For example, the e-commerce platform stores the sales data of sample merchants as part of the sample characteristics, and this part of the sample characteristics constitutes the above-mentioned first characteristic matrix; the electronic payment platform maintains the merchant's transaction flow data as another part of the sample Special, constitute the second characteristic matrix. The electronic payment platform also maintains the label of the sample merchant (whether it is a fraudulent merchant or not), which constitutes a label vector Y.

In other scenario examples, the business object may also be other objects to be evaluated, such as commodities, interaction events (for example, transaction events, login events, click events, purchase events), and so on. Correspondingly, the participating parties may be different business parties that maintain different characteristic parts of the above-mentioned business objects. The business prediction model may be a model that performs classification prediction or regression prediction for the corresponding business object.

It needs to be understood that the business object features maintained by both parties belong to private data. During the joint training process, plaintext exchanges are not allowed to protect the security of private data. And, finally, the first party A wants to train to obtain the model parameter part used to process the first feature part, called the first parameter part W _A ; the second party wants to train to obtain the second parameter part used to process the second feature part W _B , these two parts of parameters together constitute a business forecasting model.

In order to conduct joint training of the model without leaking private data, according to the embodiment of this specification, as shown in FIG. 1, the first party A and the second party B will initialize the first parameter part W to be trained _A and the second parameter part W _B are secretly shared and disassembled into parameter fragments, so the first party obtains the first parameter first fragment <W _A > ₁ and the second parameter first fragment <W _B > ₁ , The second party obtains the second segment of the first parameter <W _A > ₂ and the second segment of the second parameter <W _B > ₂ .

_{In the iterative training process of the model, both parties obtain the encrypted fragments Z 1} , Z ₂ of the product result of the total feature matrix X and the total parameter matrix W through the security matrix multiplication. The second party with the label sums up the two encrypted fragments, and obtains the encrypted product result Z. The second party obtains the encrypted error vector E based on the product result Z and the encrypted label vector Y, and performs secret sharing under homomorphic encryption. Therefore, both parties obtain error fragments E ₁ and E _{2 respectively} . _{Further, the two parties obtain the corresponding gradient fragments G 1} and G ₂ through secret sharing and security matrix multiplication based on the error fragments and their respective feature matrices. Then, the first party uses its gradient segment G ₁ to update its maintained parameter segments <W _A > ₁ and <W _B > ₁ , and the second party uses its gradient segment G ₂ to update its maintained parameter segments <W _A > ₂ and <W _B > ₂ .

Until the end of the entire iterative process, the two parties exchange their parameter fragments and perform parameter reconstruction. Therefore, the first party reconstructs the first parameter part after training based on the first parameter first fragment <W _A > ₁ maintained by itself and the second parameter second fragment <W _A > _{2 sent by the second party} W _a; second party based on a second parameter which is maintained by a second fragment <W _{B> 2} and the second parameter of the first party sends a first fragment <W _{B> 1,} to give a second reconstructed training The parameter part W _B.

During the entire training process, not only did the two parties not exchange the feature data in plaintext, but the model parameters were also split into parameter shards, and each only maintained the iterative update of the sharding parameters. The model parameters would not be reconstructed until the end of the iteration. In this way, the security of private data in the joint training process is greatly enhanced.

It can be seen that in the above training methods, a secret sharing scheme under homomorphic encryption and a secure matrix multiplication scheme are needed. The two schemes are briefly described below.

Figure 2 shows a secret sharing scheme under homomorphic encryption in one embodiment. In the example scenario in Figure 2, the first party A owns the public key PK-a and the corresponding private key SK-a for homomorphic encryption, and the second party B owns the public key PK-b and the corresponding private key SK- b. Assume that the matrix Z is currently to be secretly shared, and the matrix Z has been homomorphically encrypted with the public key PK-a of the first party A.

In the context of this article, square brackets [] are used to indicate encryption, and the superscripts indicate the public key used for encryption. In this way, the matrix to be shared is denoted as [Z] _a .

In order to secretly share the homomorphic encrypted matrix [Z] _a , the second party B randomly generates a second fragment <Z> ₂ locally.

In the context of this article, angle brackets <> are used to indicate the secretly shared fragment, and the corner mark indicates the holder of the fragment.

Then, the second party B uses the public key PK-a of the first party A and the same homomorphic encryption algorithm to encrypt the second fragment <Z> ₂ to obtain the second encrypted fragment [<Z> ₂ ] _a .

Next, the second party B _{performs a homomorphic subtraction operation on the matrix [Z] a} and the second encrypted fragment [<Z> ₂ ] _a to obtain the first encrypted fragment [<Z> ₁ ] _a = [Z] _a- [<Z> ₂ ] _a .

Here, the homomorphism of the homomorphic encryption algorithm is used, that is, the operation of the plaintext is performed before encryption, and the corresponding operation of the ciphertext after encryption is performed, and the result is equivalent. For example, using the same public key PK to encrypt v ₁ and v _{2 to} obtain E _PK (v ₁ ) and E _PK (v ₂ ), if it satisfies:

Then it is considered that the encryption algorithm satisfies the additive homomorphism, where

Add operation for the corresponding homomorphism. Practice,

Operations can correspond to regular addition, multiplication, etc. For example, in Paillier's algorithm,

Corresponds to regular multiplication.

The above calculation of the homomorphic subtraction in the first encrypted segment is the corresponding subtraction operation of the homomorphic addition operation.

Then, the second party B sends the above-mentioned first encrypted fragment [<Z> ₁ ] _a to the first party A. Since the first encrypted fragment is encrypted using the public key of the first party A, the first party can decrypt it with the corresponding private key SK-a to obtain the first fragment <Z> ₁ .

Therefore, in the end, the first party A owns the first shard <Z> ₁ , and the second party B owns the second shard <Z> ₂ , and according to the above homomorphism, the sum of the two shards is the original matrix Z : <Z> ₁ +<Z> ₂ =Z. In this way, the secret sharing between the two parties under homomorphic encryption is realized.

Figure 3 shows the implementation of secure matrix multiplication in one embodiment. In the example scenario in Figure 3, the first party A owns the matrix X and the second party B owns the matrix Y. Both parties hope to jointly calculate the product matrix X*Y without revealing their respective matrix plaintexts. To this end, a secure matrix multiplication based on homomorphic encryption can be used.

Specifically, the first party A can use its public key PK-a to encrypt its original matrix X using a homomorphic encryption algorithm to obtain an encryption matrix [X] _a , and send the encryption matrix to the second party B.

The second party B performs the homomorphic summation between the ciphertext elements in the encrypted matrix [X] _a and the plaintext elements in the matrix Y to obtain an encrypted product matrix [Z] _a = [X] _a * Y. According to the homomorphism of the encryption algorithm, the encrypted product matrix [Z] _a corresponds to the matrix obtained by encrypting the product matrix X*Y of the original matrices X and Y using the public key PK-a of Party A using the homomorphic encryption algorithm. That is, [Z] _a =[X*Y] _a .

Then, the above-mentioned encrypted product matrix [Z] _{a is} used as the homomorphic encryption matrix [Z] _a to be shared in Figure 2, and the secret sharing under homomorphic encryption is performed. So in the end, the first party A owns the first shard <Z> ₁ , and the second party B owns the second shard <Z> ₂ , and the sum of the two shards is the product matrix X*Y: <Z> ₁ +<Z> ₂ =X*Y.

In this way, a secure matrix multiplication between the two parties is realized.

It needs to be understood that Figure 3 is an implementation example of secure matrix multiplication. There are other secure matrix multiplication implementations, such as matrix multiplication based on secret sharing, etc., which will not be detailed here.

Using secret sharing under homomorphic encryption and secure matrix multiplication, the joint training of the model shown in Figure 1 can be realized. The following describes the specific process of the two parties jointly conducting model training.

Fig. 4 shows a schematic diagram of a process of joint training of a linear regression model by two parties according to an embodiment. The data holding status of the first party A and the second party B in the scenario of FIG. 4 is the same as that of FIG. 1, and will not be repeated here. In addition, the first party A and the second party B can send their own public keys PK-a and PK-b to each other. In the scenario in Figure 4, the two parties jointly train a linear regression model as a business prediction model.

First, in the model initialization stage, the first party A and the second party B initialize the model parameters and share secretly, each maintaining parameter slicing.

Specifically, in step S11, the first party for processing the first initialization parameter A W _A portion of the first feature section. The first parameter may be initialized W _A portion obtained by way of randomly generated. Then, at S12, the first party A secretly shares the above-mentioned first parameter part, that is, splits it into the first parameter first segment <W _A > ₁ and the first parameter second _{segment <W A} > ₂ , Hold the first segment of the first parameter <W _A > ₁ and send the second segment of the first parameter <W _A > ₂ to the second party B. It can be understood that the sum of the two parameter fragments is the first parameter part, namely: W _A =<W _A > ₁ +<W _A > ₂ .

Correspondingly, in step S13, the second party B initializes the second parameter part W _B for processing the second characteristic part. The second parameter part W _B can be initialized in a randomly generated manner. Then, in S14, the second party A secretly shares the above-mentioned second parameter part, and splits it into the second parameter first segment <W _B > ₁ and the second parameter second _{segment <W B} > ₂ , Holds the second parameter second fragment <W _B > ₂ and sends the second parameter first fragment <W _B > ₁ to the first party A. Correspondingly, the sum of these two parameter fragments is the second parameter part, namely: W _B =<W _B > ₁ +<W _B > ₂ .

It should be understood that steps S11-S12 and steps S13-S14 can be executed in parallel or in any order, which is not limited here.

After the above initialization and secret sharing, the first party A maintains the first parameter first fragment <W _A > ₁ and the second parameter first fragment <W _B > ₁ , and the second party B maintains the first parameter The second segment <W _A > ₂ and the second parameter of the second segment <W _B > ₂ .

Next, enter the model iteration stage, which generally includes multiple iterations. In one embodiment, the number of iterations is a preset hyperparameter. In another embodiment, the number of iterations is not preset, but the iteration is stopped when a certain convergence condition is met. The above convergence conditions may be, for example, that the error is low enough, the gradient is small enough, and so on.

Each iteration process can include 4 sub-phases: calculate the product of the total feature matrix X and the total parameter W; calculate the error vector E; calculate the gradient G; update the parameters. The following describes the specific implementation of each sub-phase.

In the first sub-stage, in step S21, the first party A and the second party B respectively calculate the first multiplication integral piece <Z> ₁ and the second multiplication based on the local matrix multiplication operation and the safety matrix multiplication operation of both parties. integral sheet <Z> _2, such that the two fragments corresponds to the total sum of the product of the feature matrix X parameter W, which is equal to a first feature matrix X _a portion of the first product of the first parameter multiplied W _a , And the sum of the second product of the second feature matrix X _B and the second parameter part W _B.

Fig. 5 shows part of the implementation process of the first sub-stage in one embodiment.

Specifically, in step S211, the first party A locally calculates the product of the first feature matrix X _A and the first segment of the first parameter <W _A > ₁ to obtain the first feature first processing result <Z _A > ₁ , that is :

<Z _A > ₁ = X _A ^˙ <W _A > ₁

In step S212, the first party A uses the first feature matrix X _{A held by the first party A to} perform a security matrix multiplication with the first parameter second slice <W A> _{2 held by the second party B.} The safe matrix multiplication can be implemented in the manner shown in Figure 3, or implemented in other safe calculation methods. The product of the first feature matrix X _A and the second segment of the first parameter <W _A > ₂ is recorded as the first feature second processing result <Z _A > ₂ , namely:

<Z _A > ₂ = X _A ^˙ <W _A > ₂

In the context of this article, the result of processing with local parameters is referred to as the first processing result, and the result of processing with the other party's parameters through secure matrix multiplication is referred to as the second processing result.

Then through the security matrix multiplication in step S212, the first party A obtains the first feature of the second processing result <Z _A > ₂ of the first fragment <<Z _A > ₂ > ₁ , and the second party B obtains the first feature of the second The second segment of the processing result <Z _A > _{2 is <<Z A} > ₂ > ₂ , and the sum of the two segments is the second processing result of the first feature.

In step S213, the second party B locally calculates the product of the second feature matrix X _B and the second parameter second segment <W _B > ₂ to obtain the first processing result of the second feature <Z _B > ₁ , namely:

<Z _B > ₁ = X _B ^˙ <W _B > ₂

In step S214, the second party B uses the second feature matrix X _{B held by the second party B to} perform the security matrix multiplication with the second parameter first slice <W B> ₁ held by the first party A, and the product is denoted as second The second processing result of the feature <Z _B > ₂ , namely:

<Z _B > ₂ = X _B ^˙ <W _B > ₁

Through the security matrix multiplication in step S214, the first party A obtains the first segment of the second feature second processing result <Z _B > ₂ <<Z _B > ₂ > ₁ , and the second party B obtains the second feature second processing The second fragment of the result <Z _B > _{2 <<Z B} > ₂ > ₂ , the sum of the two fragments is the second processing result of the second feature.

It should be understood that the above steps S211-S214 can be performed in any order.

Then, in step S215, the first party A adds up the pieces of the processing results obtained by the above calculations, that is, the first processing result of the first feature <Z _A > ₁ , the second processing result of the first feature The first segment <<Z _A > ₂ > ₁ , the first segment of the second processing result of the second feature <<Z _B > ₂ > ₁ is added to obtain the first multiplied integral <Z> ₁ , namely:

<Z> ₁ =<Z _A > ₁ +<<Z _A > ₂ > ₁ +<<Z _B > ₂ > ₁

Correspondingly, in step S216, the second party B adds up the pieces of each processing result obtained by it, that is, the second piece of the second processing result of the first feature <<Z _A > ₂ > ₂ , The first processing result of the second feature <Z _B > ₁ , and the second segment of the second processing result of the second feature <<Z _B > ₂ > ₂ is added to obtain the second multiplication-integral segment <Z> ₂ , namely:

<Z> ₂ =<Z _B > ₁ +<<Z _A > ₂ > ₂ +<<Z _B > ₂ > ₂

It can be verified that the sum of the first multiplying integral piece <Z> ₁ and the second multiplying integral piece <Z> ₂ is the product of the total feature matrix X and the total parameter W, that is, the first feature matrix X _A and the first parameter part The _{sum of the first product of W A} and the second product of the second feature matrix X _B and the second parameter part W _B :

<Z> ₁ +<Z> ₂

＝<Z _A > ₁ +<<Z _A > ₂ > ₁ +<<Z _B > ₂ > ₁ +<Z _B > ₁ +<<Z _A > ₂ > ₂ +<<Z _B > ₂ > ₂

＝<Z _A > ₁ +(<<Z _A > ₂ > ₁ +<<Z _A > ₂ > ₂ )+<Z _B > ₁ +(<<Z _B > ₂ > ₁ +<<Z _B > ₂ > ₂ )

＝X _A ^˙ <W _A > ₁ +X _A ^˙ <W _A > ₂ +X _B ^˙ <W _B > ₁ +X _B ^˙ <W _B > ₂

＝X _A ^˙ W _A +X _B ^˙ W _B

So far, the first party A and the second party B have calculated the first multiplying integral piece <Z> ₁ and the second multiplying integral piece <Z> _{2 respectively} .

Go back to the first sub-stage in Figure 4. Since the second party B owns the tag data, and in order to protect the privacy of the data, in step S22 of Figure 4, the first party A uses its public key PK-a to homomorphically encrypt the _{above-mentioned first multiplier piece <Z> 1} , Get the first encrypted multiplication integral piece [<Z> ₁ ] _a , and send the first encrypted multiplication integral piece [<Z> ₁ ] _a to the second party B.

In step S23, the second party B also uses the public key PK-a of the first party to _{perform homomorphic encryption on the second multiplier <Z> 2} obtained by calculation to obtain the second encrypted multiplier [<Z > ₂ ] _a .

Then, in step S24, the second party B performs a homomorphic summation on the first encrypted multiplication integral piece [<Z> ₁ ] _a and the second encrypted multiplication integral piece [<Z> ₂ ] _a to obtain the encrypted product result [Z ] _a :

[Z] _a =[<Z> ₁ ] _a +[<Z> ₂ ] _a

According to the homomorphism of the encryption algorithm and the relationship between the first multiplier integral piece and the second multiplier integral piece, the encrypted product result [Z] _a obtained in this way corresponds to the first characteristic matrix X _A and the first parameter part The encrypted value of the sum of the first product of W _A and the second product of the second characteristic matrix X _B and the second parameter part W _{B , namely [X A} ^˙ W _A +X _B ^˙ W _B ] _a , In other words, the encrypted value of the product of the total feature matrix X and the total parameter W.

In this way, in the first sub-stage of the iteration, the second party B obtains the encrypted product result [Z] _a through the security calculation performed by both parties, which corresponds to the encrypted value of the product of the total feature matrix X and the total parameter W. Then, enter the second sub-stage, and calculate the error vector E.

In step S31 of the second sub-stage, the second party B _{performs a homomorphic operation based on the encrypted product result [Z] a} and the encrypted value of the label vector Y to obtain the encrypted error vector [E] _a .

In the scenario of the linear regression model shown in Figure 4, the predicted value

Therefore, the prediction error

It can be expressed as the difference between the product result X*W of the feature matrix and the model parameters and the label vector Y. The product result currently obtained is in the encrypted form [Z] _a . Therefore, the label vector Y can be homomorphically encrypted first to obtain [Y] _a , and then the encrypted product result [Z] _a and the label vector encryption value [Y] ] of _a difference with the state, as an encryption error vector [E] _a, namely:

[E] _a = [Z] _a -[Y] _a

_{Then, in step S32, the encryption error vector [E] a} is secretly shared using, for example, the secret sharing under homomorphic encryption as shown in FIG. 2. Through this secret sharing, the first party A obtains the first error fragment <E> ₁ , the second party B obtains the second error fragment <E> ₂ , and <E> ₁ +<E> ₂ =E.

Then, enter the third sub-phase of the iteration to calculate the gradient. According to the previous formula (1), the gradient calculation involves the multiplication of the error vector and the feature matrix. However, the error vector and the feature matrix are still distributed between the first party A and the second party B. Therefore, a piecewise calculation method is still needed to obtain each gradient piece.

Specifically, in step S41, the second party B locally _{performs matrix multiplication under the homomorphic operation on the encryption error vector [E] a} and the second eigen matrix X _B to obtain the second encryption gradient [G _B ] _a , namely :

[G _B ] _a =[E] _a ^T˙ X _B

Wherein, [E] _a ^T represents [E] _a transposition, and the operations between _a ^T and X _B [E], is [E] _a ^T ciphertext elements X _B each row expressly each column The homomorphic addition operation between elements is similar to the homomorphic matrix multiplication in the secure matrix multiplication process in Figure 3.

Then, at step S42, the second party on the second B encryption gradient [G _{B] a} secret sharing at the homomorphic encryption, for example, FIG. 2 is used. Through this secret sharing, the first party A obtains the second gradient first fragment <G _B > ₁ , and the second party B obtains the second gradient second fragment <G _B > ₂ , and the sum of the fragments is the second gradient G _B = E ^T˙ X _B.

In step S43, the first party A _{performs a local multiplication operation on the transposition of the first error segment <E> 1} and the first feature matrix X _A to obtain the first part of the first gradient <G _A > ₁ , namely:

<G _A > ₁ ＝<E> ₁ ^T˙ X _A

The above operations are local operations of the first party.

Then, in step S44, the first party uses the first feature matrix X _{A to} perform a safety matrix multiplication with the second error slice <E> ₂ in the second party, and the result of the multiplication is recorded as the first gradient second part< G _A > ₂ , namely:

<G _A > ₂ ＝<E> ₂ ^T˙ X _A

Through the above security matrix multiplication, the first party A gets the first slice of the second part of the first gradient<<G _A > ₂ > ₁ , and the second party B gets the second slice of the second part of the first gradient <<G _A > ₂ > ₂ .

So far, the calculation of gradient slicing is realized. Then, enter the fourth sub-phase of the iteration, parameter update. In this stage, each party updates the parameter shards maintained by themselves according to the gradient shards obtained by themselves. The parameter update phase includes the following steps.

In step S51, the first portion of the first gradient of the first party A calculated according to step S43 <G _{A> 1} obtained in step S44 and the first slice << G _A second portion of the first gradient>_{2> 1,} the first update One parameter first fragment <W _A > ₁ .

Specifically, the product of the sum of the first part of the first gradient <G _A > ₁ and the first slice of the second part of the first gradient <<G _A > ₂ > ₁ and the preset step size α is used as the adjustment amount, and the Subtract the adjustment amount, update the first parameter, the first slice <W _A > ₁ , which can be expressed as:

<W _A > ₁ ←<W _A > ₁ -α(<G _A > ₁ +<<G _A > ₂ > ₁ )

In step S52, the first party A updates the second parameter first fragment <W _B > ₁ according to the second gradient first fragment <G _B > ₁ obtained in step S42, which can be expressed as:

<W _B > ₁ ←<W _B > ₁ -α<G _B > ₁

In step S53, the second party B updates the first parameter and the second _{segment <W A} > _{2 according to the second segment <<G A} > ₂ > _{2 of the} second part of the first gradient obtained in step S44, which can mean for:

<W _A > ₂ ←<W _A > ₂ -α<<G _A > ₂ > ₂

In step S54, the second party B updates the second parameter second segment <W _B > ₂ according to the second gradient second segment <G _B > ₂ obtained in step S42, which can be expressed as:

<W _B > ₂ ←<W _B > ₂ -α<G _B > ₂

That is, on the basis of the original slice value, the product of the preset step size α and the corresponding gradient slice is subtracted, thereby updating each parameter slice. It can be understood that the above steps S51-S54 can be executed in any order, or executed in parallel.

It can be seen that _{the update of the first parameter part W A} is jointly completed by both parties, where the first party A updates the first parameter first fragment <W _A > ₁ , and the second party B updates the first parameter second fragment < W _A > ₂ , the sum of the two parties' common update is:

<G _A > ₁ +<<G _A > ₂ > ₁ +<<G _A > ₂ > ₂

＝<G _A > ₁ +<G _A > ₂

＝<E> ₁ ^T˙ X _A +<E> ₂ ^T˙ X _A

＝E ^T˙ X _A

That is, the product of (transpose of) the error vector and the first feature matrix X _A.

The update of the second parameter part W _B is also done by both parties. The first party A updates the second parameter first _{segment <W B} > ₁ , and the second party B updates the second parameter second _{segment <W B} > _2. The sum of the two parties' joint updates is:

<G _B > ₁ +<G _B > ₂

＝G _B ＝E ^T˙ X _B

That is, the product of the error vector (transpose of) and the second feature matrix X _B.

However, after each round of iteration, the two parties do not need to exchange updated parameter fragments, but continue to the next iteration, that is, return to step S21, and execute the first sub-phase again based on the updated parameter fragments. In this way, in the iterative process, neither party has complete model parameters, nor does it exchange the plaintext information of the feature matrix, which ensures the security of private data with high strength.

Until the end of the entire iteration process, for example, the preset number of iterations is reached, or the predetermined convergence condition is reached, the model reconstruction phase is entered.

In the model reconstruction phase, the first party A sends its iteratively maintained second parameter first fragment <W _B > ₁ to the second party B; the second party B will iteratively maintain the first parameter second fragment <W _A > _{2 is} sent to the first party A.

The first party A reconstructs the first parameter part after training based on the first parameter first fragment <W _A > ₁ maintained by itself and the first parameter second fragment <W _A > _{2 sent by the second party} W _A.

Based on the second parameter second fragment <W _B > ₂ maintained by the second party itself and the second parameter first fragment <W _B > ₁ sent by the first party, the second parameter part after training is reconstructed W _B.

Thus, the first party the second party A and B have completed the training of the linear regression model, respectively, to give each model parameter section W _A and W _B used to treat the corresponding characteristic portion.

Looking back at the entire training process, it can be seen that the two parties not only do not exchange the plaintext of the feature data, but also split the model parameters into parameter shards, and each only maintains the iterative update of the sharding parameters. The model will not be reconstructed until the end of the iteration. parameter. In the iterative process, all parties only maintain parameter shards and exchange some sharding results, and it is almost impossible to infer useful information about private data based on these sharding results. This greatly enhances the privacy data in the joint training process. safety.

The joint training of the linear regression model in Figure 4 is described in detail above. The following describes the scenario of the logistic regression model. Those skilled in the art understand that when a logistic regression model is used as a business prediction model, the predicted value can be expressed as:

It can be seen that the predicted value of the logistic regression model is based on the non-linear sigmoid function, and the non-linear function is not conducive to secure calculations such as homomorphic encryption.

Therefore, in the case of a logistic regression model, in order to facilitate linear calculation, the sigmoid function can be expanded by Taylor Taylor. Specifically, the sigmod function 1/(1+e^x) can perform the following Taylor decomposition:

Correspondingly, the predicted value of logistic regression can be expanded into:

Substituting the above predicted value expansion into formula (1), the gradient form can be obtained. For example, under the first-order expansion, the gradient form is

The gradient form of the third-order expansion is

In this way, through Taylor Taylor expansion, the predicted value of logistic regression is converted into a scheme that can use homomorphic encryption. Therefore, the program process shown in Figure 4 can be slightly modified to make the training process suitable for the logistic regression model.

Fig. 6 shows a schematic diagram of a process of joint training of a logistic regression model by two parties according to an embodiment. The training process of Figure 6 is basically the same as that of Figure 4, except that in step S31, when calculating the encryption error vector, according to the Taylor expansion form of the sigmoid function _{, the encrypted prediction result is obtained based on the encrypted product result [Z] a} , and the encrypted prediction result and label vector The encrypted value of Y is subjected to homomorphic difference operation, and the encrypted error vector E is obtained.

In the case of adopting the first-order Taylor expansion, according to formula (4), the prediction result can be expressed as (0.5+Z/4), and the error term can be divided into (0.5-Y) and Z/4 accordingly. Therefore, the approximate encryption error vector [E] _a under logistic regression can be obtained through the following operations:

[E] _a = [0.5-Y] _a -[Z] _a /4

The other training steps are the same as in Figure 4.

In the case of using multi-order Taylor expansion, it is also necessary to further obtain the multi-order calculation result of wx, that is, the encrypted value [Z ^k ] _{a of the} ^{multi-order product result Z k} . When calculating the encryption error vector [E] _a , the encrypted prediction result is obtained based on the encrypted product result [Z] _a and the encrypted multi-factor product [Z ^k ] _a , and the homomorphic difference between the encrypted prediction result and the encrypted value of the label vector Y is performed Only by calculation can the encryption error vector be obtained.

Specifically, for example, in the case of adopting the third-order expansion, that is, k=3, it is necessary to further obtain [Z ³ ] _a . For this reason, on the basis of the first multiplying integral piece <Z> ₁ and the second multiplying integral piece <Z> ₂ obtained by both parties in S21 in Fig. 6, high-order calculations and result exchanges can be performed to obtain [Z ³ ] _a . ^{For example, the encrypted value [Z 3} ] _{a of the} result of the third-order product can be calculated by the following formula.

After that, the encryption error vector [E] _{a can be calculated based on the homomorphic operation of [Z] a} , [Z ³ ] _a and the encrypted label vector Y according to formula (5).

It can be understood that the higher the order of Taylor expansion, the more accurate the result, but the higher the computational complexity. But in principle, the high-order product result can be calculated based on the low-order shards. In this way, for the business prediction model implemented by the logistic regression model, the two-party joint training to protect data privacy can be realized through the method described above.

The above training methods are also applicable to business prediction models implemented by neural networks. For a typical feedforward fully connected neural network, each neuron is connected to each neuron in the previous layer with different weights. Therefore, the output of each neuron in the previous layer can be regarded as feature data, and the feature data is distributed between the two sides; the connection weight can be regarded as the model parameter part, which is used to process the corresponding feature data in a linear combination. Therefore, the aforementioned training process can be applied to the parameter training of each neuron in the neural network to realize the joint safety training of the two parties of the neural network model.

In general, for various business prediction models based on linear combinations of feature data and model parameters, the training methods described above can be used. In this training method, through the fragmented maintenance of parameters, high strength ensures that private data will not be leaked or reversed, and data security is ensured.

According to another embodiment, there is provided a device for two parties to jointly train a service prediction model to protect data privacy. The two parties include a first party and a second party, and the device can be deployed in the second party. Wherein, the first party stores a first feature matrix X _A composed of first feature parts of multiple business objects; the second party stores a second feature matrix X _B composed of second feature parts of the multiple business objects, And the label vector Y formed by the label value. The second party can be implemented as any device, platform or device cluster with computing and processing capabilities. Fig. 7 shows a schematic block diagram of a joint training device deployed in a second party according to an embodiment. As shown in FIG. 7, the device 700 includes an iterative unit 710 for performing model parameter update multiple iterations. The iteration unit 710 further includes:

The multiplication-integral piece calculation unit 711 is configured to calculate the homomorphism based on the locally maintained first parameter second piece and the second parameter second piece through local matrix multiplication and safe matrix multiplication with the first party The encrypted second encrypted multiplying integral piece, and receiving the first encrypted multiplying integral piece from the first party; wherein the first parameter second piece is used to process the first parameter part W _{A of the first characteristic part} The second segment of the second parameter; the second segment of the second parameter is the second segment used to process the second parameter part W _B of the second characteristic part;

The product result determination unit 712 is configured to perform a homomorphic summation on the first encrypted multiplying integral piece and the second encrypted multiplying integral piece to obtain an encrypted product result Z, which corresponds to the first characteristic matrix X _A and the first parameter The encrypted value of the sum of the first product of the part W _A and the second product of the second feature matrix X _B and the second parameter part W _B;

The error vector determining unit 713 is configured to perform a homomorphic operation based on the encrypted product result Z and the encrypted value of the tag vector Y to obtain an encrypted error vector E, and secretly share the encrypted error vector E to obtain a second error score piece;

The first gradient determining unit 714 is configured to perform matrix multiplication under the homomorphic operation on the encryption error vector E and the second characteristic matrix X _B to obtain the second encryption gradient, and perform secret sharing on the second encryption gradient to obtain the first Two-gradient second slice;

The second gradient determining unit 715 is configured to use the second error _{segment to perform a security matrix multiplication with the first feature matrix X A} in the first party to obtain a second segment of the second part of the first gradient;

The parameter update unit 716 is configured to update the second parameter second slice according to the second slice of the second gradient; update the first parameter according to the second slice of the second part of the first gradient The second fragment.

In an embodiment, the above-mentioned apparatus 700 further includes an initialization unit 720 configured to:

Initialize the second parameter part W _B , split it into a second parameter first fragment and a second parameter second fragment through secret sharing, retain the second parameter second fragment, and divide the second parameter The first fragment of the parameter is sent to the first party;

Receiving a first secret parameter sharing part W _A second fragment of the first parameter from the first party.

According to an implementation manner, the above-mentioned apparatus 700 further includes a parameter reconstruction unit 730, configured to: send the second segment of the first parameter updated in the last iteration to the first party, and receive the update from the first party. One party receives the updated first segment of the second parameter;

Combine the updated second parameter second segment in the last iteration with the received second parameter first segment to obtain the second parameter part W _B after the service prediction model is trained.

In different embodiments, the foregoing business objects include one of the following: users, merchants, commodities, and events; the business prediction model is used to predict the classification or regression value of the business objects.

In a specific embodiment, the service prediction model is a linear regression model; at this time, the error vector determining unit 713 is configured to calculate the homomorphic difference between the encrypted product result Z and the label vector Y as the Encryption error vector E.

In another specific embodiment, the service prediction model is a logistic regression model; at this time, the error vector determining unit 713 is configured to obtain an encrypted prediction result based on the encrypted product result Z according to the Taylor expansion form of the sigmoid function, and The encrypted prediction result and the encrypted value of the label vector Y are subjected to a homomorphic difference operation to obtain the encrypted error vector E.

Further, in an example, the product result determining unit 712 is further configured to calculate the encrypted multi-order product at least according to the first multiplying integral piece and the second multiplying integral piece; correspondingly, the error vector determining unit 713 is configured To obtain an encrypted prediction result based on the encrypted product result Z and the encrypted multi-order product according to the multi-order Taylor expansion form of the sigmoid function, and perform a homomorphic difference operation on the encrypted prediction result and the encrypted value of the label vector Y , The encryption error vector E is obtained.

In a specific embodiment, the above-mentioned multiply-integral piece calculation unit 711 is specifically configured to: use the first parameter second piece to perform a safe matrix multiplication _{with the first feature matrix X A in the first party to obtain the first} A second segment of the second processing result of a feature; locally calculating the product of the second feature matrix X _B and the second segment of the second parameter to obtain the first processing result of the second feature; using the second feature matrix X _B , Perform a security matrix multiplication with the first segment of the second parameter in the first party to obtain the second segment of the second processing result of the second feature; for the second segment of the second processing result of the first feature, The first processing result of the second feature, the second segment of the second processing result of the second feature are added, and the addition result is homomorphically encrypted with the public key of the first party to obtain the first Two encrypted multiplying integral pieces.

In a specific example, the above parameter update unit 716 is configured to update the second parameter second slice by subtracting the product of the second gradient second slice and the preset step size.

According to another embodiment, there is provided a device for two parties to jointly train a business prediction model. The device can be deployed in the aforementioned first party, and the first party can be implemented as any device or platform with computing and processing capabilities. Or device cluster. As mentioned above, the first party stores the first feature matrix X _A formed by the first feature parts of the multiple business objects; the second party stores the second features formed by the second feature parts of the multiple business objects Matrix X _B , and label vector Y composed of label values. Fig. 8 shows a schematic block diagram of a joint training device deployed in a first party according to an embodiment. As shown in FIG. 8, the device 800 includes an iterative unit 810 for performing model parameter update multiple iterations. The iteration unit 810 further includes:

The multiply-integral piece calculation unit 811 is configured to calculate the same value based on the locally maintained first parameter first piece and the second parameter first piece through the local matrix multiplication operation and the secure matrix multiplication operation with the second party. multiply encrypting the encrypted first state integral piece; wherein the first parameter is a first fragment of the first fragment processing a first portion of the first characteristic parameter W _a portion; a second parameter of the first minutes A slice is the first slice used to process the second parameter part W _B of the second characteristic part;

The multiplying integral piece sending unit 812 is configured to send the first encrypted multiplying integral piece to the second party, so that the second party performs a homomorphic summation of the first encrypted multiplying integral piece and the second encrypted multiplying integral piece calculated by the first encrypted multiplying integral piece, encryption result to obtain a product Z, which corresponds to a first product of the first feature matrix X _a W _a portion of the first parameter multiplied, and the second feature matrix X _B W _B with the second parameter multiplied by a second portion The encrypted value of the sum of products;

The error fragment receiving unit 813 is configured to receive the first error fragment secretly shared with the encrypted error vector E from the second party, wherein the encrypted error vector E is based on the homomorphism of the encrypted product result Z and the encrypted value of the label vector Y Operational determination;

The first gradient determining unit 814 is configured to _{perform a local multiplication operation on the transposition of the first error segment and the first feature matrix X A} to obtain the first part of the first gradient;

The second gradient determining unit 815 is configured to use the first feature matrix X _{A to} perform a security matrix multiplication with the second error segment retained in the second party to obtain the first segment of the second part of the first gradient;

The third gradient determining unit 816 is configured to receive, from the second party, the second gradient first fragment that is secretly shared with the second encrypted gradient;

The parameter update unit 817 is configured to update the first parameter first slice according to the first slice of the first part of the first gradient and the first slice of the second part of the first gradient; update the first slice according to the first slice of the second gradient The first segment with two parameters.

In one embodiment, the device 800 further includes an initialization unit 820 configured to: initialize the first parameter part W _A , and split it into a first parameter first segment and a first parameter second segment through secret sharing. , Reserve the first fragment of the first parameter, and send the second fragment of the first parameter to the second party; receive from the second party the second parameter first that is secretly shared with the second parameter _{part W B} Fragmentation.

According to an embodiment, the device 800 further includes a parameter reconstruction unit 830, configured to: send the first fragment of the second parameter updated in the last iteration to the second party, and from the second party The party receives the updated first parameter second segment; the updated first parameter first segment in the last iteration is combined with the received first parameter second segment to obtain the service prediction model training After the first parameter part W _A.

According to a specific embodiment, the multiplication-integral piece calculation unit 811 is specifically configured to: locally calculate the product of the first feature matrix X _A and the first piece of the first parameter to obtain the first processing result of the first feature; using the first feature The matrix X _A is multiplied by a security matrix with the first parameter second slice in the second party to obtain the first slice of the first characteristic second processing result; the first slice with the second parameter is used with Perform security matrix multiplication on the second feature matrix X _B in the second party to obtain the first segment of the second processing result of the second feature; for the first feature of the first processing result, the first feature is the second The first segment of the processing result is added, and the first segment of the second processing result of the second feature is added, and the result of the addition is homomorphically encrypted with the public key of the first party to obtain the first encryption Multiply the integral piece.

In one embodiment, the above-mentioned parameter update unit 817 is at least configured to take the product of the sum of the first part of the first gradient and the first part of the second part of the first gradient and the preset step length as the adjustment amount, and The first segment of the first parameter is updated by subtracting the adjustment amount.

Through the above devices deployed in the first party and the second party, the security joint training of the two parties to protect data privacy is realized.

According to another embodiment, there is also provided a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed in a computer, the computer is caused to execute the method described in conjunction with FIG. 4 to FIG. 5.

According to an embodiment of still another aspect, there is also provided a computing device, including a memory and a processor, the memory stores executable code, and when the processor executes the executable code, a combination of FIGS. 4 to 5 is provided. The method described.

Those skilled in the art should be aware that, in one or more of the above examples, the functions described in the present invention can be implemented by hardware, software, firmware, or any combination thereof. When implemented by software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on the computer-readable medium.

The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific embodiments of the present invention, and are not intended to limit the scope of the present invention. The protection scope, any modification, equivalent replacement, improvement, etc. made on the basis of the technical solution of the present invention shall be included in the protection scope of the present invention.

Claims (22)

1. A method for two parties to jointly train a business prediction model to protect data privacy. The two parties include a first party and a second party. The first party stores a first feature matrix X composed of first feature parts of multiple business objects. _{A ; the second party stores a second feature matrix X B} composed of the second feature parts of the multiple business objects, and a label vector Y composed of label values; the method is applied to the second party, the The method includes performing model parameter update multiple iterations, where each iteration includes:

   Based on the locally maintained first parameter second fragment and the second parameter second fragment, by using the matrix multiplication operation directly performed locally, and using the security performed between the second party and the first party Matrix multiplication operation, the second encrypted multiplication integral piece of homomorphic encryption is obtained by calculation, and the first encrypted multiplication integral piece is received from the first party; wherein the second piece of the first parameter is used to process the first feature the second fragment of the first part of the parameter W _a portion; a second parameter of the second slice is the second slice for processing the second portion of the second characteristic parameter W _B of the portion;

   A first integrator by the first encryption and the second encryption sheet by sheet integral homomorphic summed and the result is encrypted product Z, which corresponds to the first feature matrix X _A W _A and the first parameter section multiplied Product, and the encrypted value of the sum of the second product of the second feature matrix X _B and the second parameter part W _B;

   Perform a homomorphic difference operation based on the encrypted product result Z and the encrypted value of the label vector Y to obtain an encrypted error vector E, and secretly share the encrypted error vector E to obtain a second error fragment;

   Perform matrix multiplication under homomorphic operation on the encryption error vector E and the second characteristic matrix X _B to obtain the second encryption gradient, and perform secret sharing on the second encryption gradient to obtain the second gradient second slice;

   Use the second error fragment to _{perform a security matrix multiplication operation with the first feature matrix X A} in the first party to obtain the second fragment of the second part of the first gradient;

   According to the second segment of the second gradient, update the second segment of the second parameter; update the second segment of the first parameter according to the second segment of the second part of the first gradient.
2. The method according to claim 1, before performing model parameter update multiple iterations, further comprising:

   Initialize the second parameter part W _B , split it into a second parameter first fragment and a second parameter second fragment through secret sharing, retain the second parameter second fragment, and divide the second parameter The first fragment of the parameter is sent to the first party;

   Receiving a first secret parameter sharing part W _A second fragment of the first parameter from the first party.
3. The method according to claim 1, after performing model parameter update multiple iterations, further comprising:

   Sending the updated second segment of the first parameter in the last iteration to the first party, and receiving the updated first segment of the second parameter from the first party;

   Combine the updated second parameter second segment in the last iteration with the received second parameter first segment to obtain the second parameter part W _B after the service prediction model is trained.
4. The method according to claim 1, wherein the business object includes one of the following: users, merchants, commodities, and events; and the business prediction model is used to predict the classification or regression value of the business object.
5. The method according to claim 1, wherein the business prediction model is a linear regression model;

   The performing a homomorphic difference operation based on the encrypted product result Z of the encrypted product Z and the encrypted value of the label vector Y to obtain the encrypted error vector E includes:

   The homomorphic difference between the encrypted product result Z and the label vector Y is calculated as the encrypted error vector E.
6. The method according to claim 1, wherein the business prediction model is a logistic regression model;

   The performing a homomorphic difference operation based on the encrypted product result Z of the encrypted product Z and the encrypted value of the label vector Y to obtain the encrypted error vector E includes:

   According to the Taylor expansion form of the sigmoid function, an encrypted prediction result is obtained based on the encrypted product result Z, and a homomorphic difference operation is performed on the encrypted prediction result and the encrypted value of the tag vector Y to obtain the encrypted error vector E.
7. The method according to claim 6, wherein before obtaining the encryption error vector E, further comprising: calculating an encrypted multi-order product at least according to the first multiplying integral piece and the second multiplying integral piece;

   The obtaining of the encryption error vector E includes:

   According to the multi-order Taylor expansion form of the sigmoid function, the encrypted prediction result is obtained based on the encrypted product result Z and the encrypted multi-order product, and the homomorphic difference operation is performed on the encrypted prediction result and the encrypted value of the label vector Y to obtain The encryption error vector E.
8. The method according to claim 1, wherein calculating the second encrypted multiplication integral piece of homomorphic encryption comprises:

   Use the first parameter second segment to _{perform a security matrix multiplication operation with the first feature matrix X A} in the first party to obtain the second segment of the first feature second processing result;

   Locally calculating the product of the second feature matrix X _B and the second segment of the second parameter to obtain the first processing result of the second feature;

   Use the second feature matrix X _{B to} perform a security matrix multiplication operation with the first segment of the second parameter in the first party to obtain the second segment of the second processing result of the second feature;

   Add the second segment of the second processing result of the first feature, the first processing result of the second feature, and the second segment of the second processing result of the second feature, and use the first party Perform homomorphic encryption on the addition result with the public key of to obtain the second encrypted multiplication integral piece.
9. The method according to claim 1, wherein, according to the second gradient second fragment, updating the second parameter second fragment includes: subtracting the second gradient second fragment from a preset The product of the step size, the second segment of the second parameter is updated.
10. A method for two parties to jointly train a business prediction model to protect data privacy. The two parties include a first party and a second party. The first party stores a first feature matrix X composed of first feature parts of multiple business objects. _{A ; the second party stores a second feature matrix X B} composed of the second feature parts of the multiple business objects, and a label vector Y composed of label values; the method is applied to the first party, the The method includes: multiple iterations to perform model parameter update, where each iteration includes:

    The first shard based on the first parameter and the first shard of the second parameter maintained locally, by using the matrix multiplication operation directly performed locally, and the security performed between the first party and the second party matrix multiplication operation, the first encryption calculated by integrating homomorphic encryption substrate; wherein the first parameter of the first fragment is a first fragment for a first characteristic portion of the first process parameter W _a portion of ; The first fragment of the second parameter is the first fragment used to process the second parameter part W _B of the second characteristic part;

    Send the first encrypted multiplication integral piece to the second party, so that the second party homomorphically sums the first encrypted multiplication integral piece and the second encrypted multiplication integral piece calculated to obtain the encrypted product result Z, which corresponds to in the first feature with the first parameter matrix X _a W _a portion of the first product of multiplication, and the second feature matrix X _B and the encrypted value and a second product portion of the second parameter multiplied W _B;

    Receiving, from the second party, the first error fragment secretly shared with the encryption error vector E, where the encryption error vector E is determined based on the homomorphic difference operation of the encrypted product result Z and the encrypted value of the label vector Y;

    Performing a local multiplication operation on the transposition of the first error _{segment and the first feature matrix X A} to obtain the first part of the first gradient;

    Use the first feature matrix X _{A to} perform a security matrix multiplication operation with the second error segment retained in the second party to obtain the first segment of the second part of the first gradient;

    Receiving, from the second party, the second gradient first segment that is secretly shared with the second encryption gradient;

    According to the first slice of the first part of the first gradient and the first slice of the second part of the first gradient, update the first slice of the first parameter; according to the first slice of the second gradient, update the first slice of the second parameter.
11. The method according to claim 10, before performing the model parameter update for multiple iterations, further comprising:

    Initializing the first parameter part W _A, a secret shared by a first parameter which is split into a first slice and a second slice of the first parameter, the first parameter of the first retention fragments, the first The second fragment of the parameter is sent to the second party;

    Receive the first fragment of the second parameter secretly shared with the second parameter _{part W B} from the second party.
12. The method according to claim 10, after performing model parameter update for multiple iterations, further comprising:

    Sending the updated first segment of the second parameter in the last iteration to the second party, and receiving the updated second segment of the first parameter from the second party;

    The updated after the last iteration of the first slice of the first parameter, the first parameter and the second slice the received combination parameters to obtain the first portion of the rear of the train traffic prediction model W _A.
13. The method according to claim 10, wherein calculating the first encrypted multiplication integral piece of homomorphic encryption comprises:

    Locally calculating the product of the first feature matrix X _A and the first segment of the first parameter to obtain the first processing result of the first feature;

    Using the first feature matrix X _{A to} perform a security matrix multiplication operation with the second segment of the first parameter in the second party to obtain the first segment of the second processing result of the first feature;

    Use the first segment with the second parameter to _{perform a security matrix multiplication operation with the second feature matrix X B} in the second party to obtain the first segment with the second processing result of the second feature;

    The first processing result of the first feature, the first segment of the second processing result of the first feature, and the first segment of the second processing result of the second feature are added, and the first segment is used. Perform homomorphic encryption on the addition result with the public key of to obtain the first encrypted multiplication integral piece.
14. The method according to claim 10, wherein, according to the first slice of the first part of the first gradient and the first slice of the second part of the first gradient, updating the first slice of the first parameter comprises: changing the first slice of the first parameter The product of the sum of the first segment of the first part of the gradient and the second part of the first gradient and the preset step size is used as the adjustment amount, and the first parameter first segment is updated by subtracting the adjustment amount.
15. A device for two parties to jointly train a business prediction model to protect data privacy. The two parties include a first party and a second party. The first party stores a first feature matrix X composed of first feature parts of multiple business objects. _{A ; the second party stores a second feature matrix X B} composed of the second feature parts of the multiple business objects, and a label vector Y composed of tag values; the device is deployed on the second party, the The device includes an iterative unit for performing model parameter update multiple times, and further includes:

    The multiplication-integral piece calculation unit is configured to be based on the locally maintained first parameter second piece and the second parameter second piece, by adopting the matrix multiplication operation directly executed locally, and adopting the method between the second party and the The secure matrix multiplication operation performed between the first party is calculated to obtain the second encrypted multiplication integral piece of homomorphic encryption, and the first encrypted multiplication integral piece is received from the first party; wherein, the second piece of the first parameter is _{The second slice of the first parameter part W A} used to process the first characteristic part; the second slice of the second parameter is the second slice used to process the second parameter part W _B of the second characteristic part piece;

    The product result determining unit is configured to perform a homomorphic summation on the first encrypted multiplying integral piece and the second encrypted multiplying integral piece to obtain an encrypted product result Z, which corresponds to the first characteristic matrix X _A and the first parameter part The encrypted value of the sum of the first product of W _A and the second product of the second feature matrix X _B and the second parameter part W _B;

    The error vector determining unit is configured to perform a homomorphic difference operation based on the encrypted product result Z and the encrypted value of the label vector Y to obtain an encrypted error vector E, and secretly share the encrypted error vector E to obtain a second error Fragmentation;

    The first gradient determining unit is configured to perform matrix multiplication under the homomorphic operation on the encryption error vector E and the second characteristic matrix X _B to obtain a second encryption gradient, and perform secret sharing of the second encryption gradient to obtain a second encryption gradient. Gradient second slice;

    The second gradient determining unit is configured to use the second error _{segment to perform a security matrix multiplication operation with the first feature matrix X A} in the first party to obtain the second segment of the second part of the first gradient;

    The parameter update unit is configured to update the second parameter second slice according to the second slice of the second gradient; update the first parameter second slice according to the second slice of the second part of the first gradient Two slices.
16. The device according to claim 15, further comprising an initialization unit configured to:

    Initialize the second parameter part W _B , split it into a second parameter first fragment and a second parameter second fragment through secret sharing, retain the second parameter second fragment, and divide the second parameter The first fragment of the parameter is sent to the first party;

    Receiving a first secret parameter sharing part W _A second fragment of the first parameter from the first party.
17. The device according to claim 15, further comprising a parameter reconstruction unit, configured to: send the second fragment of the first parameter updated in the last iteration to the first party, and send the second fragment from the first The party receives the updated first segment of the second parameter;

    Combine the updated second parameter second segment in the last iteration with the received second parameter first segment to obtain the second parameter part W _B after the service prediction model is trained.
18. A device for two parties to jointly train a business prediction model to protect data privacy. The two parties include a first party and a second party. The first party stores a first feature matrix X composed of first feature parts of multiple business objects. _{A ; the second party stores a second feature matrix X B} composed of the second feature parts of the multiple business objects, and a label vector Y composed of tag values; the device is deployed on the first party, the The device includes: an iterative unit for performing model parameter update multiple times, and further includes:

    The multiplying-integral piece calculation unit is configured to be based on the first piece of the first parameter maintained locally and the first piece of the second parameter, by adopting the matrix multiplication operation directly executed locally, and adopting the method between the first party and the The secure matrix multiplication operation performed between the second party is calculated to obtain the first encrypted multiplication integral piece of homomorphic encryption; wherein, the first parameter first piece is the first parameter used to process the first characteristic part W _a first slicing portion; a second parameter for processing a first fragment of the first fragment of the second parameter part W _B of the second characteristic portion;

    The multiplying integral piece sending unit is configured to send the first encrypted multiplying integral piece to the second party, so that the second party performs a homomorphic summation of the first encrypted multiplying integral piece and the second encrypted multiplying integral piece calculated by the first encrypted multiplying integral piece to encryption obtained multiplication result Z, which corresponds to a first product of the first feature matrix X _a W _a portion of the first parameter multiplied, and a second product of the second feature matrix X _B W _B with the second parameter section multiplied The encrypted value of the sum;

    The error fragment receiving unit is configured to receive the first error fragment secretly shared with the encrypted error vector E from the second party, wherein the encrypted error vector E is based on the homomorphic difference between the encrypted product result Z and the encrypted value of the label vector Y Value calculation is determined;

    A first gradient determining unit, configured to _{perform a local multiplication operation on the transposition of the first error segment and the first feature matrix X A} to obtain the first part of the first gradient;

    The second gradient determining unit is configured to use the first feature matrix X _{A to} perform a security matrix multiplication operation with the second error segment retained in the second party to obtain the first segment of the second part of the first gradient;

    The third gradient determining unit is configured to receive, from the second party, the second gradient first fragment that is secretly shared with the second encrypted gradient;

    The parameter update unit is configured to update the first segment of the first parameter according to the first segment of the first part of the first gradient and the first segment of the second part of the first gradient; update the second segment of the first parameter according to the first segment of the second gradient The first segment of the parameter.
19. The device according to claim 18, further comprising an initialization unit configured to:

    Initializing the first parameter part W _A, a secret shared by a first parameter which is split into a first slice and a second slice of the first parameter, the first parameter of the first retention fragments, the first The second fragment of the parameter is sent to the second party;

    Receive the first fragment of the second parameter secretly shared with the second parameter _{part W B} from the second party.
20. The device according to claim 18, further comprising a parameter reconstruction unit configured to:

    Sending the updated first segment of the second parameter in the last iteration to the second party, and receiving the updated second segment of the first parameter from the second party;

    The updated after the last iteration of the first slice of the first parameter, the first parameter and the second slice the received combination parameters to obtain the first portion of the rear of the train traffic prediction model W _A.
21. A computer-readable storage medium having a computer program stored thereon, and when the computer program is executed in a computer, the computer is caused to execute the method according to any one of claims 1-14.
22. A computing device, comprising a memory and a processor, characterized in that executable code is stored in the memory, and when the processor executes the executable code, the method described in any one of claims 1-14 is implemented. method.

Images