WO2021092980A1

WO2021092980A1 - Longitudinal federated learning optimization method, apparatus and device, and storage medium

Info

Publication number: WO2021092980A1
Application number: PCT/CN2019/119418
Authority: WO
Inventors: 范涛; 杨恺; 陈天健; 杨强
Original assignee: 深圳前海微众银行股份有限公司
Priority date: 2019-11-14
Filing date: 2019-11-19
Publication date: 2021-05-20
Also published as: CN110851786B; CN110851786A

Abstract

The present application relates to the technical field of fintech. Disclosed are a longitudinal federated learning optimization method, apparatus and device, and a storage medium. The method comprises: a subsidiary participant acquiring an encrypted value set having a linear regression value and sent by a master participant, and calculating subsidiary encrypted data according to the encrypted value set; sending the subsidiary encrypted data to a coordinator, wherein the coordinator is used to update, in response to the case where a longitudinal federated model does not converge, a second-order derivative matrix in the coordinator according to the subsidiary encrypted data, and calculate a target subsidiary gradient value according to the updated second-order derivative matrix; and receiving the target subsidiary gradient value sent by the coordinator on the basis of the subsidiary encrypted data, updating local model parameters in the subsidiary participant on the basis of the target subsidiary gradient value, and continuing to execute the step of the subsidiary participant acquiring the encrypted value set having a linear regression value and sent by the master participant until the longitudinal federated model corresponding to the coordinator converges.

Description

Longitudinal federation learning optimization method, device, equipment and storage medium

Technical field

This application relates to the technical field of financial technology (Fintech), in particular to vertical federated learning optimization methods, devices, equipment, and storage media.

Background technique

With the development of computer technology, more and more technologies (big data, distributed, blockchain, artificial intelligence, etc.) are applied in the financial field. The traditional financial industry is gradually transforming to Fintech. However, due to financial The industry's security and real-time requirements also place higher requirements on technology. For example, the longitudinal linear regression method in federated learning, the existing longitudinal linear regression scheme is a stochastic gradient descent method based on one-step information. Take the vertical federated linear regression in which two parties participate as an example. For example, if party A and party B are set, party A is the host (host) party, with only part of the characteristics of the data, and party B is the guest (guest) party, which has part and A completeness. Different data characteristics have data tags at the same time. Party B needs to request the inner product of the current model parameters and data from Party A to calculate the loss function value and gradient. This process involves Party A sending its encrypted calculation data to Party B, and Party B calculates the encrypted coefficients. , Through the coefficients AB, the two parties can calculate their respective gradient components, and send them to the third party C for decryption and processing, and then send them back to A and B as the descending direction, update the model parameters held by both parties, and iterate this step So A and B can get a trained model. The existing schemes are iteratively optimized based on one-step information of the objective loss function, and its convergence speed is slow. This results in a large number of rounds of data interaction between ABCs, and communication takes a lot of time in cross-enterprise cooperation.

Summary of the invention

The main purpose of this application is to propose a vertical federated learning optimization method, device, equipment, and storage medium, which aims to solve the current long-term technical problem of vertical federated learning.

In order to achieve the above objective, the present application provides a longitudinal federated learning optimization method. The longitudinal federated learning optimization method includes the following steps:

The secondary participant obtains the encrypted value set with linear regression value sent by the main participant, and calculates the secondary encrypted data according to the encrypted value set;

The loss function value and the secondary encrypted data are sent to the coordinator, where the coordinator is used to update the second derivative matrix in the coordinator according to the secondary encrypted data in response to the vertical federation model not converging , And calculate the target subgradient value according to the updated second derivative matrix;

Receive the target sub-gradient value sent by the coordinator based on the sub-encrypted data, update the local model parameters in the sub-participant based on the target sub-gradient value, and continue to execute the sub-participant's acquisition of the data sent by the main participant The step of gathering the encrypted values of the linear regression value until the vertical federation model corresponding to the coordinator converges.

In addition, this application also provides a longitudinal federated learning optimization method including the following steps:

Receive the primary encrypted data sent by the primary participant and the secondary encrypted data sent by the secondary participant, wherein the secondary encrypted data is calculated according to the intermediate result value in the secondary participant, and the intermediate result value is the secondary The participant calculates based on the encrypted value set sent by the main participant, and the encrypted value set includes the main encrypted value and the new encrypted value;

In response to the failure of the longitudinal logistic regression model to converge, update a second derivative matrix according to the main encrypted data and the auxiliary encrypted data, and calculate a target auxiliary gradient value according to the updated second derivative matrix;

The target secondary gradient value is sent to the secondary participant, and the secondary participant is used to update the local model parameters in the secondary participant based on the target secondary gradient value, and continue to perform the secondary participant acquisition The step of collecting encrypted values with linear regression values sent by the main participant until the vertical federation model corresponding to the coordinator converges.

In addition, in order to achieve the above-mentioned purpose, the present application also provides a longitudinal federated learning optimization device, the longitudinal federated learning optimization device includes:

The obtaining module is used for the secondary participant to obtain the encrypted value set with linear regression value sent by the main participant, and calculate the secondary encrypted data according to the encrypted value set;

The sending module is configured to send the secondary encrypted data to the coordinator, where the coordinator is used to update the second derivative matrix in the coordinator according to the secondary encrypted data in response to the vertical federation model not converging, And calculate the target sub-gradient value according to the updated second derivative matrix;

The first receiving module is configured to receive the target secondary gradient value sent by the coordinator based on the secondary encrypted data, update the local model parameters in the secondary participant based on the target secondary gradient value, and continue to execute the secondary participant The step of obtaining the encrypted value set with linear regression value sent by the main participant until the vertical federation model corresponding to the coordinator converges.

Optionally, the longitudinal federated learning optimization device further includes:

The second receiving module is used to receive the primary encrypted data sent by the primary participant and the secondary encrypted data sent by the secondary participant, wherein the secondary encrypted data is calculated according to the intermediate result value in the secondary participant, and the The intermediate result value is calculated by the secondary participant based on the encrypted value set sent by the main participant, and the encrypted value set includes the main encrypted value and the new encrypted value;

The update module is configured to respond to the failure of the longitudinal logistic regression model to converge, update the second derivative matrix according to the main encrypted data and the auxiliary encrypted data, and calculate the target sub-gradient value according to the updated second derivative matrix;

The convergence module is configured to send the target secondary gradient value to the secondary participant, and the secondary participant is used to update the local model parameters in the secondary participant based on the target secondary gradient value and continue to execute all The step of obtaining the encrypted value set with linear regression value sent by the main participant by the secondary participant until the vertical federation model corresponding to the coordinator converges.

In addition, in order to achieve the above object, this application also provides a longitudinal federated learning optimization device, the longitudinal federated learning optimization device includes: a memory, a processor, and a computer stored on the memory and capable of running on the processor A readable instruction, when the computer readable instruction is executed by the processor, implements the steps of the vertical federated learning optimization method as described above.

In addition, in order to achieve the above-mentioned object, the present application also provides a storage medium having computer-readable instructions stored on the storage medium, and when the computer-readable instructions are executed by a processor, the above-mentioned vertical federated learning optimization method is implemented. step.

Description of the drawings

FIG. 1 is a schematic diagram of a device structure of a hardware operating environment involved in a solution of an embodiment of the present application;

FIG. 2 is a schematic flowchart of the first embodiment of the vertical federated learning optimization method according to this application;

FIG. 3 is a schematic flowchart of another embodiment of the vertical federated learning optimization method of this application;

Figure 4 is a schematic diagram of the device modules of the vertical federated learning optimization device of the application;

Figure 5 is a schematic diagram of the calculation and interaction process of the vertical federated learning optimization method of this application.

The realization, functional characteristics, and advantages of the purpose of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

Detailed ways

It should be understood that the specific embodiments described here are only used to explain the application, and not used to limit the application.

As shown in FIG. 1, FIG. 1 is a schematic diagram of the device structure of the hardware operating environment involved in the solution of the embodiment of the present application. The longitudinal federated learning optimization device in the embodiment of the present application may be a PC or a server device, on which a Java virtual machine runs. As shown in FIG. 1, the vertical federated learning optimization device may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to implement connection and communication between these components. The user interface 1003 may include a display screen (Display) and an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as a magnetic disk memory. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001. Those skilled in the art can understand that the structure of the device shown in FIG. 1 does not constitute a limitation on the device, and may include more or fewer components than those shown in the figure, or a combination of certain components, or different component arrangements. As shown in FIG. 1, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and computer readable instructions. In the device shown in FIG. 1, the network interface 1004 is mainly used to connect to the back-end server and communicate with the back-end server; the user interface 1003 is mainly used to connect to the client (user side) and communicate with the client; and the processor 1001 can be used to call computer-readable instructions stored in the memory 1005, and perform operations in the following vertical federated learning optimization method.

Based on the above hardware structure, an embodiment of the longitudinal federated learning optimization method of this application is proposed. Referring to Fig. 2, Fig. 2 is a schematic flowchart of a first embodiment of a longitudinal federated learning optimization method according to this application. The method includes:

Step S10, the secondary participant obtains the encrypted value set with linear regression value sent by the main participant, and calculates the secondary encrypted data according to the encrypted value set;

Linear regression is a method based on a linear model to fit data features (independent variables) and data labels (dependent variables). Vertical federated linear regression means that multiple participants want to combine data for linear regression modeling, but each holds a part of different data characteristics, and the data labels are often owned by only one party. Therefore, in this embodiment, the main participant has only part of the characteristics of the data, while the sub-participants have some data characteristics that are completely different from the main participant. Linear regression (linear regression) model training is to minimize the loss function L(w)=∑ _i ||w ^T x _i -y _i || ² _{under given data features and labels (x i} , y _i) The process of obtaining model parameters ω. Vertical federated learning means that different parties have different feature data, which is equivalent to dividing each complete data into multiple parts vertically. Each party hopes to implement linear regression model training while protecting data privacy, so as to use the model The parameter predicts the value of the dependent variable on the new data. This scheme adopts an encryption method satisfying additive homomorphism, namely [[ax]]=a[[x]],[[x]]+[[y]]=[[x+y]]. Here [[·]] stands for homomorphic encryption operation. In a vertical federation scenario, only one party holds data labels. Take two parties as an example. Party A holds data x _A and maintains the corresponding model parameters w _A , and Party B holds x _B , y _B and owns and maintains the corresponding model parameters. w _B.

In order to achieve longitudinal federated linear regression, it is necessary to calculate the loss function value and gradient, respectively: loss=l(w)=||w ^T xy|| ² ,

The loss function and gradient can be expressed as the operation of the homomorphic encrypted data of both parties, namely:

This solution uses second-order information to propose a fast-convergent technical solution, based on the second-order derivative matrix of the loss function (ie, the Hessian matrix)

The design idea of this scheme is based on the quasi-Newton method, using the second-order information to estimate an inverse Hessian matrix H. In the algorithm, the gradient g is not used but H _{g is} used as the descending direction to speed up the convergence speed of the algorithm. Since the dimension of the inverse Hessian matrix H is much larger than the gradient, the core point of the design is how to reduce the data communication volume of all parties. This scheme proposes to maintain the inverse Hessian matrix H at the C end, and in addition to calculating the gradient for each L step AB, a small batch of data is randomly selected, and the average value of the previous L step model is calculated

Average with the last L-step model

Difference

Then calculate a vector containing the second-order information of the batch of data

Sent to the C end, the dimension is the same as the gradient. The C terminal uses the information of the first M vectors v to update the inverse Hessian matrix once. Therefore, in this embodiment, the main participant is regarded as Party A, the sub-participants are regarded as Party B, and the coordinator is regarded as Party C. Therefore, first randomly select a small batch of data ID as S, and Party A calculates the value set of the corresponding data ID in S

A uses homomorphic encryption technology for all

Encrypt the value to get the encrypted data set

Transmit it to Party B. Then, update

And judge the relationship between the current iteration number k and L, if the current iteration number k is an integer multiple of L, and the iteration number k is greater than 2L;

And calculate the current (t) and last (t-1)

Difference between

In addition, a small batch of data ID is randomly selected as S _H , and the A side calculates the value on S _H

And homomorphically encrypted data

Transmitted to the B side. If the current number of iterations k is an integer multiple of L, and the number of iterations k is not greater than 2L: only update the A side

Party B calculates the value set of the corresponding data ID in S

Party B uses the properties of homomorphic encryption to calculate the encrypted loss (loss function value) value, namely

At the same time, the encrypted value of each corresponding data [[d]]=2([[u _A ]]+[[u _B ]]+[[-y]]) is calculated and transmitted to the A terminal. Then, update

And judge the relationship between the current iteration number k and L. If the current number of iterations k is an integer multiple of L, and the number of iterations k is greater than 2L: end B is updated

And calculate the current (t) and last (t-1)

Difference between

In addition, the B side calculates the S _H

To calculate

And transmitted to the A side. If the current number of iterations k is an integer multiple of L, and the number of iterations k is not greater than 2L: only update the B end

Step S20: Send the secondary encrypted data to the coordinator, where the coordinator is used to update the second derivative matrix in the coordinator according to the secondary encrypted data in response to the vertical federation model not converging, and according to The updated second-order derivative matrix calculates the target sub-gradient value;

The two parties of AB each use the property of homomorphic encryption to multiply each [[d]] value by the corresponding data x _A , x _B , and then sum the resulting vector set to calculate the encrypted gradient value [[g _A ]] =∑[[d]]x _A ,[[g _B ]]=∑[[d]]x _B. The B end transmits the encrypted loss value to the C end. A and B respectively transmit [[g _A ]], [[g _B ]] to C. Then, determine the relationship between the current iteration number k and L. If the current iteration number k is an integer multiple of L, and the iteration number k is greater than 2L, the two ends of A and B respectively calculate the primary encrypted data and the secondary encrypted data according to the intermediate result value [[h]] [[υ _A ]]= ∑[[h]]x _A ,[[υ _B ]]=∑[[h]]x _B , and transmit to the C party. Party C (ie, the coordinator) decrypts the received data to obtain g _A , g _B , and loss. Judge whether the longitudinal linear regression model converges according to the loss, and if it converges, send the iteration stop signal to A and B to end the algorithm. If not converged, update

And judge the relationship between the number of iterations k and 2L, if k is not greater than 2L, calculate the product of the pre-selected step size and the gradient

And transmit them to A and B respectively (that is, the target secondary gradient value is obtained, and the target secondary gradient value is sent to the main participant, and the product corresponding to the secondary participant is sent to the secondary participant). If k is greater than 2L, merge the two gradients into a long vector g, calculate the step length, the product of H and g, and split them into corresponding parts A and B and transmit them to A and B respectively, namely:

If k is an integer multiple of L, then C has also received the encrypted data [[v _A ]],[[v _B ]], which can be decrypted and combined to get

And stored in a v queue of length M. At the same time, calculate the current (t) and the last (t-1)

Difference between

Store it in the s queue of length M. If the current memory has reached the maximum storage length M, delete the first one in the queue and put the latest v and s at the end of the queue. Use the m (m not greater than M) v and s in the current memory to calculate H. The calculation method is as follows: initialize with the value at the end of the memory queue, that is, calculate

H←p[m]I, where I is the identity matrix. Then iteratively calculate the updated H from the head of the queue to the end of the queue (j=1,...,m): p[j]=1/(v[j] ^T s[j]),H←(Ip[j] s[j]v[j] ^T )H(Ip[j]v[j]s[j] ^T )+p[j]s[j]s[j] ^T.

Step S40: Receive the target secondary gradient value sent by the coordinator based on the secondary encrypted data, update the local model parameters in the secondary participant based on the target secondary gradient value, and continue to execute the secondary participant acquiring the primary participant The step of sending encrypted value sets with linear regression values until the vertical federation model corresponding to the coordinator converges.

After the secondary participant receives the target secondary gradient value sent by the coordinator, it will update the secondary participant’s own local model parameters according to the target secondary gradient value. At the same time, after the primary participant receives the target primary gradient value sent by the coordinator, The model parameters of the main participant will also be updated based on the product. That is, the two parties AB use the received unencrypted vector to update their local model parameters, namely:

And based on the updated local model parameters and formulas

To calculate the new encrypted value set with logistic regression score in the main participant again, and perform the step of obtaining the encrypted value set with linear regression value again to obtain the new loss function value, until the vertical value is determined according to the new loss function value. The federation model has converged.

For example, as shown in Figure 5, there are three parties A, B, and C performing model training, where party A is the main participant, party B is the deputy participant, and party C is the coordinator. Party A calculates and transmits data locally to Party B, which means encrypting

It is transmitted to Party B. Party B performs local calculations based on the encrypted data transmitted by Party A to obtain the encryption loss function value and encryption value, and transmits [[d]][[h]] to Party A, and both parties AB also encrypt Calculate the respective gradient values and transmit them to the C party, that is, the two parties AB send [[g _A ]],[[g _B ]][[v _A ]],[[v _B ]] to C Party B also transmits [[loss]] to Party C. Party C _{decrypts the received [[v A} ]], [[v _B ]], [[loss]] to obtain the decrypted g _A , g _B , _{loss μ A} , μ _B , and judge whether the algorithm has converged according to loss, if not, update H according to the received gradient value, calculate and transmit, that is, when k is not greater than 2L, calculate the pre-selection The product of a fixed step size and the gradient

And transmit them to A and B respectively; when k is greater than 2L, merge the two gradients into a long vector g, calculate the product of step length, H and g, and split them into corresponding parts A and B respectively Transmitted to A and B, namely:

And both parties AB update the local model parameters according to the unencrypted vector passed by party C, namely

In this embodiment, after the secondary participant obtains the encrypted value set of the main participant, the loss function value and the secondary encrypted data are calculated and sent to the coordinator, so that the coordinator can determine whether the vertical federation model has converged according to the loss function value. If it does not converge, the second derivative matrix is updated according to the secondary encrypted data, and the target secondary gradient value is calculated according to the updated second derivative matrix, and then the local model parameters of the secondary participant are updated by the target secondary gradient value, thereby avoiding The prior art for longitudinal federated learning uses a first-order algorithm to make the convergence rate slower, and a large number of rounds of data interaction are required. This reduces the amount of communication for longitudinal federated learning and improves the training time of the longitudinal federated logistic regression model. The convergence rate.

Further, based on the first embodiment of the vertical federated learning optimization method of the present application, a second embodiment of the vertical federated learning optimization method of the present application is proposed. This embodiment is a refinement of the step S10 of the first embodiment of the present application. The secondary participant obtains the encrypted value set with the linear regression value sent by the main participant, including: step a, detecting whether the vertical federation model satisfies the preset Judgment condition

In this embodiment, when the main participant sends data to the sub-participants, it is also necessary to detect whether the vertical federation model satisfies the preset determination condition, for example, to determine whether the new iteration number of the vertical federation model meets the preset number condition (such as determining the new number of iterations). Whether the number of iterations is an integer multiple of the interval of iteration steps, and whether it is greater than two times greater than the preset number). And perform different operations according to different judgment results. And perform different operations according to different test results.

In step b, if it is satisfied, the secondary participant obtains the main encrypted value and the new encrypted value sent by the main participant, and uses the main encrypted value and the new encrypted value as a linear regression sent by the main participant The encrypted value collection of the value.

In the main participant, first obtain a small batch of data, and according to the formula mentioned in the above embodiment

To calculate each logistic regression score, and encrypt these logistic regression scores using homomorphic encryption technology to obtain the main encryption value. When the vertical federation model is found to meet the preset judgment conditions, a small batch of data is obtained, and According to the formula

To calculate each logistic regression score, and encrypt these logistic regression scores using homomorphic encryption technology to obtain a new encrypted value, and use the encrypted data and the new encrypted data together as an encrypted value set with a linear regression value. It should be noted that the encrypted data and the new encrypted data are not the same and are sent to the secondary participant, that is, the secondary participant will obtain the main encrypted value and the new encrypted value sent by the main participant. However, if the vertical federation model does not meet the pre-determined conditions, the main participant only sends the main encrypted value to the sub-participants, that is, at this time, the main encrypted value is a collection of encrypted values with linear regression values.

In this embodiment, by determining that the vertical federation model satisfies the preset judgment condition, the secondary participant obtains the primary encrypted value and the secondary encrypted value sent by the primary participant, and uses them as a collection of encrypted numerical values with linear regression values. Thereby improving the training speed of the longitudinal linear regression model.

Further, the step of calculating the secondary encrypted data according to the encrypted value set includes: step c, determining whether the current iteration number corresponding to the secondary participant satisfies a preset number condition,

After obtaining the secondary encrypted value set sent by the primary participant in the secondary participant, it is also necessary to determine whether the current iteration number (that is, the number of updates) of the secondary participant’s own model meets the preset number condition, and according to different judgment results Perform different operations.

Step d, if it is satisfied, calculate the intermediate result value according to the encrypted value set, and calculate the secondary encrypted data according to the intermediate result value.

When it is judged that the current iteration number meets the preset number condition, for example, the current iteration number k is an integer multiple of L, and the iteration number k is greater than 2L: end B is updated

And calculate the current (t) and last (t-1)

Difference between

In addition, the B side calculates the S _H

To calculate the intermediate result value

It is transmitted to the A side, and at the same time, the secondary encrypted data in the secondary participant will be calculated based on the intermediate result value. That is, the two ends of A and B respectively calculate the main encrypted data and the auxiliary encrypted data according to the intermediate result value [[h]] [[υ _A ]]=∑[[h]]x _A ,[[υ _B ]]=∑ [[h]]x _B , and transmit it to the C party. But if it is not satisfied, if the current iteration number k is an integer multiple of L, and the iteration number k is not greater than 2L: only update at the B end

In this embodiment, when it is determined that the current iteration number corresponding to the secondary participant meets the preset number condition, the intermediate result value is calculated according to the encrypted value set, and the secondary encrypted data is calculated by the intermediate result value, thereby ensuring that the secondary encrypted data is obtained. The accuracy of the data.

Specifically, the step of calculating the intermediate result value according to the encrypted value set, and calculating the secondary encrypted data through the intermediate result value, includes: step e, obtaining the local model parameters in the secondary participant based on the encrypted value set The current average value, and obtain the historical average value of the preset step interval before the current average value;

After the sub-participant obtains the encrypted value set sent by the main participant, the current average value of the local model parameters in the sub-participant will also be obtained

And it is also necessary to obtain the historical average value of the preset step interval before the current average value in the secondary participants.

Step f: Calculate the difference between the current average value and the historical average value, calculate an intermediate result value according to the difference value, and calculate the secondary encrypted data according to the intermediate result value.

When the current average value and the historical average value are obtained, the difference between the two needs to be calculated, that is, to calculate the current (t) and the last (t-1)

Difference between

In addition, the B side calculates the S _H

To calculate the intermediate result value

It is transmitted to the A side, and at the same time, the secondary encrypted data in the secondary participant will be calculated based on the intermediate result value. That is, the two ends of A and B respectively calculate the main encrypted data and the auxiliary encrypted data according to the intermediate result value [[h]] [[υ _A ]]=∑[[h]]x _A ,[[υ _B ]]=∑ [[h]]x _B.

In this embodiment, the intermediate result value is calculated based on the difference between the current average value and the historical average value among the driving co-participants, and the secondary encrypted data is calculated by the intermediate result value, thereby ensuring that the secondary encrypted data is obtained. The accuracy of the data.

Further, based on any one of the first to second embodiments of the vertical federated learning optimization method of this application, a third embodiment of the vertical federated learning optimization method of this application is proposed. This embodiment is a refinement of the step S30 of the first embodiment of the present application. The step of receiving the target secondary gradient value sent by the coordinator based on the secondary encrypted data includes: step g, receiving the coordinator based on the The target sub-gradient value sent by the sub-encrypted data, wherein the target sub-gradient value is obtained by the second derivative matrix updated by the coordinator according to the target data, and the target data is in response to the failure of the longitudinal logistic regression model to converge, and It is obtained by decrypting and combining the primary encrypted data and the secondary encrypted data sent by the secondary participant when the preset judgment condition is satisfied.

When the secondary participant receives the target secondary gradient value fed back by the coordinator, it can update its own local model parameters according to the target secondary gradient value. The target secondary gradient value is when the coordinator determines that the longitudinal logistic regression model does not converge and satisfies When the judgment conditions are preset, the second derivative matrix is updated according to the target data, and calculated according to the updated second derivative matrix, where the target data does not converge in the longitudinal logistic regression model and meets the preset judgment When the conditions are met, it is obtained by decrypting and combining the main encrypted data sent by the main participant and the secondary encrypted data sent by the sub-participants. And judging whether the longitudinal logistic regression model satisfies the preset judgment condition, for example, judging whether the new iteration number of the longitudinal logistic regression model meets the preset number condition (such as determining whether the new iteration number is an integer multiple of the iteration step interval, and whether it is greater than Twice is greater than the preset number of times). And perform different operations according to different judgment results.

In this embodiment, by determining that the target sub-gradient value is obtained based on the target data and the updated second-order derivative matrix, and the target data is obtained by merging the main encrypted data and the sub-encrypted data, the obtained target sub-gradient is guaranteed The accuracy of the value.

Further, the step of receiving the target secondary gradient value fed back by the coordinator includes:

Step h: Receive the target subgradient value fed back by the coordinator, where the target subgradient value is obtained by splitting the first target product by the coordinator, and the first target product is based on the response to the The second-order derivative matrix updated by the longitudinal logistic regression model to satisfy the preset judgment condition, the long vector of the combination of the main gradient value sent by the main participant and the auxiliary gradient value sent by the auxiliary participant, and the preset step The product between the lengths.

When the secondary participant receives the target secondary gradient value fed back by the coordinator, it can update its own local model parameters according to the target secondary gradient value, where the target secondary gradient value is obtained by splitting the first target product by the coordinator , And the first target product is when the longitudinal logistic regression model does not converge and meets the preset judgment conditions, according to the updated second-order derivative matrix, the main gradient value sent by the main participant and the sub-gradient value sent by the sub-participant combined long The product of the vector and the preset step size.

In this embodiment, by determining that the target secondary gradient value is obtained by the coordinator splitting the first target product, and the first target product is the product of the long vector, the preset step size, and the updated second derivative matrix, thus The accuracy of the obtained target subgradient value is guaranteed.

Step k, receiving the target subgradient value fed back by the coordinator, wherein the target subgradient value is a second target product, and the second target product is that the coordinator does not converge in response to the longitudinal logistic regression model, and The product of the calculated secondary gradient value sent by the secondary participant and the preset step size is not met.

When the sub-participant receives the target sub-gradient value fed back by the coordinator, he can update his own local model parameters according to the target sub-gradient value. The target sub-gradient value is the second product, and the second product is the longitudinal logic of the coordinator. When the regression model does not converge and the preset judgment conditions are not met, the sub-gradient value sent by the sub-participants and the preset step length are calculated to obtain its product. This product is the second product, which is the target sub-product. The gradient value.

In this embodiment, by determining the target secondary gradient value when the longitudinal logistic regression model does not converge and does not meet the preset judgment conditions, the product of the preset step size and the secondary gradient value is calculated, thereby ensuring the obtained target The accuracy of the main gradient value.

Further, referring to Fig. 3, Fig. 3 is a schematic flow chart of another embodiment of the vertical federated learning optimization method of this application, including: step S100, receiving the primary encrypted data sent by the primary participant and the secondary encrypted data sent by the secondary participant, where The secondary encrypted data is calculated according to the intermediate result value in the secondary participant, the intermediate result value is calculated by the secondary participant according to the encrypted value set sent by the main participant, the encrypted value set Including the main encrypted value and the new encrypted value;

In the coordinator, when it is determined that the longitudinal logistic regression model has not converged according to the loss function value sent by the secondary participant and meets the preset judgment condition, for example, it is judged whether the new iteration number of the longitudinal logistic regression model meets the preset number condition (such as determining the new number of iterations). Whether the number of iterations is an integer multiple of the iteration step interval, and whether it is greater than twice the preset number), if the preset number condition is met, it is determined that the longitudinal logistic regression model satisfies the preset judgment condition, when the main participant sends After the primary encrypted data and the secondary encrypted data sent by the secondary participant, the second derivative matrix is updated according to the primary encrypted data and the secondary encrypted data. Among them, the secondary encrypted data is calculated by the secondary participant based on the intermediate result value of the target value set feedback sent by the main participant, that is, the primary participant sends the encrypted value set to the secondary participant, and the secondary participant calculates it based on the encrypted value set The intermediate result value and the loss function value, and the loss function value is sent to the coordinator, the secondary encrypted data is calculated based on the intermediate result value, and the secondary encrypted data is sent to the coordinator. Among them, the encrypted value set may include the main encrypted value corresponding to the data and the new encrypted value corresponding to the new data, that is, whether the current iteration number corresponding to the main participant satisfies a preset condition (such as whether the current iteration number has passed the preset number), If it is not satisfied, the main encrypted value can be used as the target value set, and if it is satisfied, the main encrypted value and the new encrypted value can be used as the encrypted value set. And the method of data encryption in this application can be homomorphic encryption.

Step S200, in response to the failure of the longitudinal logistic regression model to converge, update a second derivative matrix according to the main encrypted data and the auxiliary encrypted data, and calculate a target subgradient value according to the updated second derivative matrix;

When the coordinator detects that the longitudinal logistic regression model has not converged, it can update the second derivative matrix based on the main encrypted data sent by the main participant and the auxiliary encrypted data sent by the sub-participants, that is, the main encrypted data and the sub-encrypted data Decrypt and merge them and store them in a queue with a preset length to obtain the target queue, and update the second derivative matrix H according to the target queue. The method of calculating H is to initialize with the value at the end of the memory queue, that is, to calculate

And in this embodiment, if the longitudinal logistic regression model does not converge, update

And need to judge the relationship between the current iteration number k and the iteration step interval L, if it is not greater than 2L, calculate the product of the step length and the gradient set in advance

And send the respective products to the corresponding party A and party B respectively, and let party A update the local model parameters of party A according to the obtained product (ie the target main gradient value), and perform the next data model training , In the same way, let Party B update the local model parameters of Party B according to the obtained product, and then perform the next model training until the new loss function value is obtained, and pass it to Party C (coordinator). Judgment, that is, to determine whether the longitudinal logistic regression model converges, and if it converges, it sends an iterative stop signal to party A and B, and stops the training of the longitudinal logistic regression model. If it does not converge, execute again

Until the longitudinal logistic regression model converges. And when k is greater than 2L, the two gradients are merged into a long vector g, the product of the step length, H and g is calculated, and split into the corresponding A and B parts (that is, the target main gradient corresponding to the A side Value and the target sub-gradient value corresponding to party B) are transmitted to A and B respectively, namely:

Step S300: Send the target secondary gradient value to the secondary participant, and the secondary participant is used to update the local model parameters in the secondary participant based on the target secondary gradient value, and continue to execute the secondary participant. The participant obtains the encrypted value set with linear regression value sent by the main participant until the vertical federation model corresponding to the coordinator converges.

After the coordinator calculates the target sub-gradient value, it will send the target sub-gradient value to the sub-participants. The sub-participants will update the local model parameters in the sub-participants according to the target sub-gradient value and continue to execute the sub-participants. The participant obtains the encrypted value set with linear regression value sent by the main participant until the longitudinal logistic regression model corresponding to the coordinator converges, and sends an iteration stop signal to the main participant and the secondary participant. Similarly, the main participant also receives the target main gradient value corresponding to the main participant fed back by the coordinator to update the local model parameters in the main participant.

In this embodiment, the coordinator updates the second-order derivative matrix according to the main encrypted data and the auxiliary encrypted data, and calculates the target sub-gradient value according to the updated second-order derivative matrix, and sends the target sub-gradient value to the sub-participants to update The local model parameters in the sub-participants, thus avoiding the phenomenon that the prior art adopts the first-order algorithm for longitudinal federated learning, which makes the convergence speed slow and requires a large number of rounds of data interaction, and reduces the communication for longitudinal federated learning. the amount.

Further, the step of updating a second derivative matrix according to the primary encrypted data and the secondary encrypted data includes:

Step m, judging whether the longitudinal logistic regression model satisfies the preset judgment condition;

After the coordinator receives the main gradient value sent by the main participant and the secondary gradient value and loss value sent by the deputy coordinator, and determines that the longitudinal logistic regression model does not converge, it needs to determine whether the longitudinal logistic regression model meets the preset determination conditions, for example Determine whether the new iteration number of the longitudinal logistic regression model meets the preset number condition (such as determining whether the new iteration number is an integer multiple of the iteration step interval, and whether it is more than twice greater than the preset number). And perform different operations according to different judgment results.

Step n, if it is satisfied, decrypt and merge the primary encrypted data and the secondary encrypted data to obtain target data;

When the longitudinal logistic regression model is found to meet the preset judgment conditions after judgment, the coordinator will decrypt and merge the target data after receiving the main encrypted data sent by the main participant and the secondary encrypted data sent by the sub-participants. Encrypted data [[v _A ]],[[v _B ]] are decrypted to obtain the target data

Step p: Store the target data in a queue with a preset length to obtain the target queue, and update the second derivative matrix through the target queue.

The coordinator stores the target data in a v queue with a length of M (ie, a preset length). At the same time, calculate the current (t) and the last (t-1)

Difference between

Store it in the s queue of length M. If the current memory has reached the maximum storage length M, delete the first one in the queue and put the latest v and s at the end of the queue. Use m (m not greater than M) v and s in the current memory to calculate H (second derivative matrix). The calculation method is as follows:

Initialize with the value at the end of the memory queue, that is, calculate

Where I is the identity matrix. Then iteratively calculate the updated H from the head of the queue to the end of the queue (j=1,...,m): p[j]=1/(v[j] ^T s[j]),H←(Ip[j] s[j]v[j] ^T )H(Ip[j]v[j]s[j] ^T )+p[j]s[j]s[j] ^T.

In this embodiment, the target data is obtained by decrypting and combining the primary encrypted data and the secondary encrypted data, and then the second derivative matrix is updated according to the target data, thereby ensuring the effectiveness of the update of the second derivative matrix.

Further, after the step of judging whether the longitudinal logistic regression model satisfies the predetermined judgment condition, the method includes:

Step x, if not satisfied, the coordinator obtains the first product between the secondary gradient value sent by the secondary participant and the preset step size, and sends the first product as the target secondary gradient value to The associate participant.

When it is judged that the longitudinal logistic regression model does not meet the preset judgment conditions, the coordinator calculates the first product of the pre-selected preset step size and the sub-gradient value, and the preset step size and the main gradient corresponding to the main participant The third product of the value, and the first product is sent to the secondary participant as the target secondary gradient value to update the local model parameters in the secondary participant, and the third product is sent to the main participant to update the local model in the primary participant Parameters, and then re-train the model according to the updated model parameters to obtain the new loss function value, and send it to the coordinator through the deputy participant.

In this embodiment, when it is determined that the longitudinal logistic regression model does not satisfy the preset determination condition, the first product between the sub-gradient value and the preset step size is calculated, and the first product is used as the target sub-gradient value, thereby The accuracy of the obtained target main gradient value is guaranteed.

The embodiment of the present application also provides a longitudinal federated learning optimization device. Referring to FIG. 4, the longitudinal federated learning optimization device includes: an acquisition module for the secondary participant to acquire the encrypted value set with linear regression value sent by the main participant , And calculate the secondary encrypted data according to the encrypted value set; the sending module is used to send the secondary encrypted data to the coordinator, wherein the coordinator is used to respond to the vertical federation model not converging, according to the secondary encryption The data updates the second-order derivative matrix in the coordinator, and calculates the target sub-gradient value according to the updated second-order derivative matrix; the first receiving module is used to receive the target sub-gradient sent by the coordinator based on the sub-encrypted data Gradient value, update the local model parameters in the secondary participant based on the target secondary gradient value, and continue to perform the step of obtaining the encrypted value set with linear regression value sent by the primary participant by the secondary participant until the coordinator The corresponding vertical federation model converges. Optionally, the acquisition module is further configured to: detect whether the vertical federation model meets a preset judgment condition; if so, the secondary participant acquires the primary encrypted value and the new encrypted value sent by the primary participant, and compares all The main encrypted value and the new encrypted value are used as an encrypted value set with a linear regression value sent by the main participant. Optionally, the acquisition module is further configured to determine whether the current iteration number corresponding to the secondary participant meets a preset number condition, and if so, calculate the intermediate result value according to the encrypted value set, and pass the The intermediate result value calculates the secondary encrypted data. Optionally, the obtaining module is further configured to: obtain the current average value of the local model parameters in the secondary participants based on the encrypted value set, and obtain the historical average of the preset step interval before the current average value Value; Calculate the difference between the current average and the historical average, and calculate an intermediate result value based on the difference, and calculate the secondary encrypted data by the intermediate result value. Optionally, the first receiving module is further configured to: receive a target secondary gradient value sent by the coordinator based on the secondary encrypted data, wherein the target secondary gradient value is updated by the coordinator according to the target data The target data is obtained by decrypting and combining the primary encrypted data and the secondary encrypted data sent by the secondary participant in response to the longitudinal logistic regression model not converging and meeting the preset judgment condition. of. Optionally, the first receiving module is further configured to: receive a target subgradient value fed back by the coordinator, wherein the target subgradient value is obtained by splitting the first target product by the coordinator The first target product is based on the second derivative matrix updated in response to the longitudinal logistic regression model satisfying the preset determination condition, the main gradient value sent by the main participant, and the main gradient value sent by the secondary participant The product of the combined long vector of the sub-gradient values and the preset step length.

Optionally, the step of receiving the target sub-gradient value fed back by the coordinator includes: receiving the target sub-gradient value fed back by the coordinator, wherein the target sub-gradient value is a second target product, and the The second target product is the calculated product between the main gradient value sent by the main participant and the preset step length calculated by the coordinator in response to the longitudinal logistic regression model not converging and not satisfying the preset determination condition. Optionally, the longitudinal federated learning optimization device further includes: a second receiving module for receiving the main encrypted data sent by the main participant and the secondary encrypted data sent by the secondary participant, wherein the secondary encrypted data is based on the The intermediate result value in the secondary participant is calculated, the intermediate result value is calculated by the secondary participant according to the encrypted value set sent by the main participant, and the encrypted value set includes the main encrypted value and the new encrypted value; The update module is configured to respond to the failure of the longitudinal logistic regression model to converge, update the second derivative matrix according to the main encrypted data and the auxiliary encrypted data, and calculate the target subgradient value according to the updated second derivative matrix; converge; Module, used to send the target secondary gradient value to the secondary participant, and the secondary participant is used to update the local model parameters in the secondary participant based on the target secondary gradient value, and continue to execute the The secondary participant obtains the encrypted value set with linear regression value sent by the main participant until the vertical federation model corresponding to the coordinator converges. Optionally, the update module is further configured to determine whether the longitudinal logistic regression model satisfies the preset determination condition; if so, decrypt and merge the primary encrypted data and the secondary encrypted data to Obtain target data; store the target data in a queue with a preset length to obtain the target queue, and update the second derivative matrix through the target queue. Optionally, the update module is further configured to: if it is not satisfied, the coordinator obtains the first product between the secondary gradient value sent by the secondary participant and the preset step size, and compares the The first product is sent to the secondary participant as the target secondary gradient value.

For the method executed by the above-mentioned program modules, please refer to the respective embodiments of the vertical federated learning optimization method of this application, and will not be repeated here.

The present application also provides a storage medium, which may be a non-volatile readable storage medium. The storage medium of the present application stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the vertical federated learning optimization method described above are realized. For the method implemented when the computer-readable instructions running on the processor are executed, please refer to the respective embodiments of the vertical federated learning optimization method of this application, which will not be repeated here.

It should be noted that in this article, the terms "include", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements not only includes those elements, It also includes other elements not explicitly listed, or elements inherent to the process, method, article, or system. Without more restrictions, the element defined by the sentence "including a..." does not exclude the existence of other identical elements in the process, method, article, or system that includes the element.

The serial numbers of the foregoing embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

Through the description of the above implementation manners, those skilled in the art can clearly understand that the above-mentioned embodiment method can be implemented by means of software plus the necessary general hardware platform, of course, it can also be implemented by hardware, but in many cases the former is better.的实施方式。 Based on this understanding, the technical solution of this application essentially or the part that contributes to the existing technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM/RAM) as described above. , Magnetic disks, optical disks), including several instructions to make a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the method described in each embodiment of the present application.

The above are only the preferred embodiments of the application, and do not limit the scope of the patent for this application. Any equivalent structure or equivalent process transformation made using the content of the description and drawings of the application, or directly or indirectly applied to other related technical fields , The same reason is included in the scope of patent protection of this application.

Claims

A longitudinal federated learning optimization method, wherein the longitudinal federated learning optimization method includes the following steps:

The secondary participant obtains the encrypted value set with linear regression value sent by the main participant, and calculates the secondary encrypted data according to the encrypted value set;

The secondary encrypted data is sent to the coordinator, where the coordinator is used to update the second derivative matrix in the coordinator according to the secondary encrypted data in response to the vertical federation model not converging, and according to the updated The second derivative matrix calculates the target subgradient value;

Receive the target secondary gradient value sent by the coordinator based on the secondary encrypted data, update the local model parameters in the secondary participant based on the target secondary gradient value, and continue to execute the secondary participant's acquisition of the primary participant's transmission Until the vertical federation model corresponding to the coordinator converges.
5. The longitudinal federated learning optimization method according to claim 1, wherein the step of obtaining the encrypted value set with linear regression value sent by the main participant by the secondary participant comprises:

Detecting that the vertical federation model meets the preset judgment condition, the secondary participant obtains the main encrypted value and the new encrypted value sent by the main participant, and sends the main encrypted value and the new encrypted value as the main participant A collection of encrypted values with linear regression values.
5. The vertical federated learning optimization method according to claim 1, wherein the step of calculating secondary encrypted data according to the encrypted value set comprises:

It is determined that the current iteration number corresponding to the secondary participant satisfies the preset number condition, then the intermediate result value is calculated according to the encrypted value set, and the secondary encrypted data is calculated based on the intermediate result value.
5. The vertical federated learning optimization method according to claim 3, wherein the step of calculating an intermediate result value according to the encrypted value set, and calculating the secondary encrypted data according to the intermediate result value, comprises:

Obtaining the current average value of the local model parameters in the secondary participants based on the encrypted value set, and obtaining the historical average value of the preset step interval before the current average value;

Calculate the difference between the current average value and the historical average value, calculate an intermediate result value based on the difference value, and calculate the secondary encrypted data based on the intermediate result value.
5. The vertical federated learning optimization method according to claim 1, wherein the step of receiving the target sub-gradient value sent by the coordinator based on the sub-encrypted data comprises:

Receive the target sub-gradient value sent by the coordinator based on the sub-encrypted data, where the target sub-gradient value is obtained by the second derivative matrix updated by the coordinator according to the target data, and the target data is in response to The longitudinal logistic regression model does not converge and meets a preset judgment condition, and is obtained by decrypting and merging the primary encrypted data and the secondary encrypted data sent by the secondary participant.
The longitudinal federated learning optimization method according to claim 1, wherein the step of receiving the target subgradient value fed back by the coordinator comprises:

Receive the target sub-gradient value fed back by the coordinator, where the target sub-gradient value is obtained by splitting the first target product by the coordinator, and the first target product is based on the response to the vertical logic The regression model satisfies the preset judgment condition and updated the second-order derivative matrix, the long vector of the combination of the main gradient value sent by the main participant and the auxiliary gradient value sent by the deputy participant, and the preset step length The product of.
The longitudinal federated learning optimization method according to claim 1, wherein the step of receiving the target subgradient value fed back by the coordinator comprises:

Receive the target sub-gradient value fed back by the coordinator, wherein the target sub-gradient value is a second target product, and the second target product is that the coordinator does not converge in response to the longitudinal logistic regression model and does not meet the expected value. Set the judgment condition, the calculated product between the main gradient value sent by the main participant and the preset step length.
A longitudinal federated learning optimization method, wherein the longitudinal federated learning optimization method includes the following steps:

Receive the primary encrypted data sent by the primary participant and the secondary encrypted data sent by the secondary participant, wherein the secondary encrypted data is calculated according to the intermediate result value in the secondary participant, and the intermediate result value is the secondary The participant calculates based on the encrypted value set sent by the main participant, and the encrypted value set includes the main encrypted value and the new encrypted value;

In response to the longitudinal logistic regression model not converging, updating a second derivative matrix according to the main encrypted data and the auxiliary encrypted data, and calculating a target auxiliary gradient value according to the updated second derivative matrix;

The target secondary gradient value is sent to the secondary participant, and the secondary participant is used to update the local model parameters in the secondary participant based on the target secondary gradient value, and continue to perform the secondary participant acquisition The step of collecting encrypted values with linear regression values sent by the main participant until the vertical federation model corresponding to the coordinator converges.
8. The longitudinal federated learning optimization method according to claim 8, wherein the step of updating a second derivative matrix according to the primary encrypted data and the secondary encrypted data comprises:

Determining that the longitudinal logistic regression model satisfies the preset determination condition, decrypting and combining the main encrypted data and the auxiliary encrypted data to obtain target data;

The target data is stored in a queue with a preset length to obtain the target queue, and the second derivative matrix is updated through the target queue.
8. The longitudinal federated learning optimization method according to claim 8, wherein the longitudinal federated learning optimization method comprises:

If it is determined that the longitudinal logistic regression model does not satisfy the preset judgment condition, the coordinator obtains the first product between the secondary gradient value sent by the secondary participant and the preset step size, and then compares the first product with the preset step size. A product is sent to the secondary participant as the target secondary gradient value.
A longitudinal federated learning optimization device, wherein the longitudinal federated learning optimization device includes:

The obtaining module is used for the secondary participant to obtain the encrypted value set with linear regression value sent by the main participant, and calculate the secondary encrypted data according to the encrypted value set;

The sending module is configured to send the secondary encrypted data to the coordinator, where the coordinator is used to update the second derivative matrix in the coordinator according to the secondary encrypted data in response to the vertical federation model not converging, And calculate the target sub-gradient value according to the updated second derivative matrix;

The first receiving module is configured to receive the target secondary gradient value sent by the coordinator based on the secondary encrypted data, update the local model parameters in the secondary participant based on the target secondary gradient value, and continue to execute the secondary participant The step of obtaining the encrypted value set with linear regression value sent by the main participant until the vertical federation model corresponding to the coordinator converges.
A longitudinal federated learning optimization device, wherein the longitudinal federated learning optimization device further includes:

The second receiving module is used to receive the primary encrypted data sent by the primary participant and the secondary encrypted data sent by the secondary participant, wherein the secondary encrypted data is calculated according to the intermediate result value in the secondary participant, and the The intermediate result value is calculated by the secondary participant based on the encrypted value set sent by the main participant, and the encrypted value set includes the main encrypted value and the new encrypted value;

The update module is configured to respond to the failure of the longitudinal logistic regression model to converge, update the second derivative matrix according to the main encrypted data and the auxiliary encrypted data, and calculate the target sub-gradient value according to the updated second derivative matrix;

The convergence module is configured to send the target secondary gradient value to the secondary participant, and the secondary participant is used to update the local model parameters in the secondary participant based on the target secondary gradient value and continue to execute all The step of obtaining the encrypted value set with linear regression value sent by the main participant by the secondary participant until the vertical federation model corresponding to the coordinator converges.
A longitudinal federated learning optimization device, wherein the longitudinal federated learning optimization device includes: a memory, a processor, and computer-readable instructions stored on the memory and running on the processor, the computer readable When an instruction is executed by the processor, the following steps are implemented:

The secondary participant obtains the encrypted value set with linear regression value sent by the main participant, and calculates the secondary encrypted data according to the encrypted value set;

The secondary encrypted data is sent to the coordinator, where the coordinator is used to update the second derivative matrix in the coordinator according to the secondary encrypted data in response to the vertical federation model not converging, and according to the updated The second derivative matrix calculates the target subgradient value;

Receive the target secondary gradient value sent by the coordinator based on the secondary encrypted data, update the local model parameters in the secondary participant based on the target secondary gradient value, and continue to execute the secondary participant's acquisition of the primary participant's transmission Until the vertical federation model corresponding to the coordinator converges.
The longitudinal federated learning optimization device according to claim 13, wherein the longitudinal federated learning optimization device comprises:

Detecting that the vertical federation model meets the preset judgment condition, the secondary participant obtains the main encrypted value and the new encrypted value sent by the main participant, and sends the main encrypted value and the new encrypted value as the main participant A collection of encrypted values with linear regression values.
A longitudinal federated learning optimization device, wherein the longitudinal federated learning optimization device includes: a memory, a processor, and computer-readable instructions stored on the memory and capable of running on the processor, the computer-readable instructions When executed by the processor, the following steps are implemented:

Receive the primary encrypted data sent by the primary participant and the secondary encrypted data sent by the secondary participant, wherein the secondary encrypted data is calculated according to the intermediate result value in the secondary participant, and the intermediate result value is the secondary The participant calculates based on the encrypted value set sent by the main participant, and the encrypted value set includes the main encrypted value and the new encrypted value;

In response to the failure of the longitudinal logistic regression model to converge, update a second derivative matrix according to the main encrypted data and the auxiliary encrypted data, and calculate a target auxiliary gradient value according to the updated second derivative matrix;

The target secondary gradient value is sent to the secondary participant, and the secondary participant is used to update the local model parameters in the secondary participant based on the target secondary gradient value, and continue to perform the secondary participant acquisition The step of collecting encrypted values with linear regression values sent by the main participant until the vertical federation model corresponding to the coordinator converges.
15. The vertical federated learning optimization device according to claim 15, wherein the step of updating a second derivative matrix according to the primary encrypted data and the secondary encrypted data comprises:

Determining that the longitudinal logistic regression model satisfies the preset determination condition, decrypting and combining the main encrypted data and the auxiliary encrypted data to obtain target data;

The target data is stored in a queue with a preset length to obtain the target queue, and the second derivative matrix is updated through the target queue.
A storage medium, wherein computer-readable instructions are stored on the storage medium, and the following steps are implemented when the computer-readable instructions are executed by a processor:

The secondary participant obtains the encrypted value set with linear regression value sent by the main participant, and calculates the secondary encrypted data according to the encrypted value set;

The secondary encrypted data is sent to the coordinator, where the coordinator is used to update the second derivative matrix in the coordinator according to the secondary encrypted data in response to the vertical federation model not converging, and according to the updated The second derivative matrix calculates the target subgradient value;

Receive the target sub-gradient value sent by the coordinator based on the sub-encrypted data, update the local model parameters in the sub-participant based on the target sub-gradient value, and continue to execute the sub-participant's acquisition of the main participant's transmission Until the vertical federation model corresponding to the coordinator converges.
17. The storage medium of claim 17, wherein the step of the secondary participant acquiring the encrypted value set with linear regression value sent by the main participant comprises:

Detecting that the vertical federation model meets the preset judgment condition, the secondary participant obtains the main encrypted value and the new encrypted value sent by the main participant, and sends the main encrypted value and the new encrypted value as the main participant A collection of encrypted values with linear regression values.
A storage medium, wherein computer-readable instructions are stored on the storage medium, and the following steps are implemented when the computer-readable instructions are executed by a processor:

Receive the primary encrypted data sent by the primary participant and the secondary encrypted data sent by the secondary participant, wherein the secondary encrypted data is calculated according to the intermediate result value in the secondary participant, and the intermediate result value is the secondary The participant calculates based on the encrypted value set sent by the main participant, and the encrypted value set includes the main encrypted value and the new encrypted value;

In response to the failure of the longitudinal logistic regression model to converge, update a second derivative matrix according to the main encrypted data and the auxiliary encrypted data, and calculate a target auxiliary gradient value according to the updated second derivative matrix;

The target secondary gradient value is sent to the secondary participant, and the secondary participant is used to update the local model parameters in the secondary participant based on the target secondary gradient value, and continue to perform the secondary participant acquisition The step of collecting encrypted values with linear regression values sent by the main participant until the vertical federation model corresponding to the coordinator converges.
19. The storage medium of claim 19, wherein the step of updating a second derivative matrix based on the primary encrypted data and the secondary encrypted data comprises:

Determining that the longitudinal logistic regression model satisfies the preset determination condition, decrypting and combining the main encrypted data and the auxiliary encrypted data to obtain target data;

The target data is stored in a queue with a preset length to obtain the target queue, and the second derivative matrix is updated through the target queue.