CN111340247A - Longitudinal federated learning system optimization method, device and readable storage medium - Google Patents
Longitudinal federated learning system optimization method, device and readable storage medium Download PDFInfo
- Publication number
- CN111340247A CN111340247A CN202010089045.6A CN202010089045A CN111340247A CN 111340247 A CN111340247 A CN 111340247A CN 202010089045 A CN202010089045 A CN 202010089045A CN 111340247 A CN111340247 A CN 111340247A
- Authority
- CN
- China
- Prior art keywords
- intermediate result
- encrypted
- equipment
- data
- simplified
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 62
- 238000005457 optimization Methods 0.000 title claims description 29
- 238000004891 communication Methods 0.000 claims abstract description 19
- 238000005070 sampling Methods 0.000 claims description 29
- 238000004364 calculation method Methods 0.000 claims description 16
- 230000009469 supplementation Effects 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 5
- 238000005056 compaction Methods 0.000 claims 1
- 238000012549 training Methods 0.000 abstract description 18
- 230000008569 process Effects 0.000 description 18
- 230000006870 function Effects 0.000 description 15
- 238000007477 logistic regression Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 5
- 238000010801 machine learning Methods 0.000 description 5
- 230000003993 interaction Effects 0.000 description 3
- 238000006467 substitution reaction Methods 0.000 description 3
- 238000012417 linear regression Methods 0.000 description 2
- 230000000717 retained effect Effects 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000004904 shortening Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
- G06N20/20—Ensemble learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Storage Device Security (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a method, equipment and a readable storage medium for optimizing a longitudinal federated learning system, wherein the method comprises the following steps: and finally, calculating by using the encrypted filling intermediate result of the second equipment to obtain an encrypted first gradient value corresponding to the model parameter in the first equipment, updating the model parameter of the first equipment based on the encrypted first gradient value, and repeating iteration until a preset stop condition is met, so as to obtain a target model parameter of the trained first equipment. In the longitudinal federal training, the data quantity required to be encrypted and communicated is reduced by reducing the data number contained in the intermediate result of the participating equipment, so that the encryption and communication cost is reduced, and meanwhile, the longitudinal federal modeling time is greatly shortened.
Description
Technical Field
The invention relates to the technical field of machine learning, in particular to a longitudinal federal learning system optimization method, device and readable storage medium.
Background
With the development of artificial intelligence, people provide a concept of 'federal learning' for solving the problem of data islanding, so that both federal parties can train a model to obtain model parameters without providing own data, and the problem of data privacy disclosure can be avoided.
In the longitudinal federated learning, under the condition that the data features of the participants are overlapped less and the users are overlapped more, the part of the users and the data with the same users and different user data features of the participants are taken out to jointly train the machine learning model. For example, there are two participants a and B belonging to the same region, where participant a is a bank and participant B is an e-commerce platform. Participants a and B have more users in the same area, but a and B have different services and different recorded user data characteristics. In particular, the user data characteristics of the a and B records may be complementary. In such a scenario, vertical federated learning may be used to help a and B build a joint machine learning predictive model, helping a and B provide better service to their customers.
In the modeling process of longitudinal federal learning, participants interact with intermediate results used for calculating gradient and loss functions in an encryption mode, each round of model training needs to encrypt and exchange each data in the intermediate results, the number of the intermediate results is the same as that of the data owned by the participants, so that the data volume of encryption and interaction is large, the encryption and communication cost is high, and meanwhile, the longitudinal federal modeling time is increased.
Disclosure of Invention
The invention mainly aims to provide a method, a device, equipment and a readable storage medium for optimizing a longitudinal federal learning system, aiming at reducing encryption and communication costs in the longitudinal federal learning training process and shortening modeling time.
In order to achieve the above object, the present invention provides a longitudinal federal learning system optimization method, which is applied to a first device participating in longitudinal federal learning, wherein the first device is in communication connection with a second device, and the longitudinal federal learning system optimization method includes the following steps:
receiving an encrypted and simplified intermediate result of the second device sent by the second device, wherein the second device is configured to sample an original intermediate result corresponding to each sample data of the second device obtained through calculation to obtain a simplified intermediate result corresponding to a part of sample data of the second device, and encrypt the simplified intermediate result of the second device to obtain an encrypted and simplified intermediate result of the second device;
performing data supplementation on the encrypted and simplified intermediate result of the second device to obtain an encrypted and supplemented intermediate result of the second device, wherein the data quantity of the encrypted and supplemented intermediate result is the same as that of the original intermediate result;
and calculating to obtain an encrypted first gradient value corresponding to the model parameter in the first equipment by using the encrypted filling intermediate result of the second equipment, updating the model parameter of the first equipment based on the encrypted first gradient value, and performing iteration circularly until a preset stopping condition is met to obtain a target model parameter of the trained first equipment.
Optionally, the step of performing data padding on the encrypted and simplified intermediate result of the second device to obtain an encrypted and padded intermediate result includes:
acquiring a sampling comparison table of the second device, and determining filling data and a filling position corresponding to the filling data in an encrypted and simplified intermediate result of the second device based on the sampling comparison table of the second device;
and inserting the filling data into the encrypted and simplified intermediate result of the second device based on the filling position to obtain an encrypted and supplemented intermediate result of the second device.
Optionally, the step of calculating an encrypted first gradient value corresponding to a model parameter in the first device by using the encrypted padding intermediate result of the second device includes:
calculating to obtain an encrypted and simplified intermediate result of first equipment for calculating the gradient value;
calculating to obtain an encrypted intermediate result of the first device by utilizing the encrypted filling intermediate result of the second device and the encrypted simplified intermediate result of the first device;
and calculating to obtain an encrypted first gradient value corresponding to the model parameter in the first equipment by using the encrypted intermediate result of the first equipment.
Optionally, the step of calculating an intermediate result of the encrypted reduction of the first device for calculating the gradient value includes:
the first equipment performs sampling processing on the original intermediate result corresponding to each sample data of the first equipment obtained through calculation to obtain a simplified intermediate result corresponding to partial sample data of the first equipment;
and encrypting the simplified intermediate result of the first device to obtain an encrypted simplified intermediate result of the first device.
Optionally, the step of calculating the encrypted intermediate result of the first device by using the encrypted padding intermediate result of the second device and the encrypted compacting intermediate result of the first device includes:
performing data complementation on the encrypted and simplified intermediate result of the first device to obtain an encrypted and complemented intermediate result of the first device;
and calculating to obtain an encrypted intermediate result of the first device by using the encrypted filling intermediate result of the first device and the encrypted filling intermediate result of the second device.
In order to achieve the above object, the present invention further provides a longitudinal federal learning system optimization method, which is applied to a second device participating in longitudinal federal learning, and the longitudinal federal learning system optimization method includes the following steps:
sampling the original intermediate result corresponding to each sample data of the second equipment obtained by calculation to obtain a simplified intermediate result corresponding to part of sample data of the second equipment;
encrypting a simplified intermediate result of second equipment to obtain a simplified intermediate result corresponding to part of sample data of the second equipment, and sending the simplified intermediate result to the first equipment, so that the first equipment feeds back the encrypted intermediate result of the first equipment based on the encrypted simplified intermediate result of the second equipment, wherein the first equipment is used for performing data complementation on the received encrypted simplified intermediate result of the second equipment to obtain an encrypted complemented intermediate result of the second equipment, and calculating by using the encrypted complemented intermediate result of the second equipment to obtain the encrypted intermediate result of the first equipment;
and calculating to obtain an encrypted second gradient value corresponding to the model parameter in the second equipment by using the encrypted intermediate result of the first equipment, updating the model parameter of the second equipment based on the encrypted second gradient value, and performing loop iteration until a preset stop condition is met to obtain a target model parameter of the trained second equipment.
Optionally, the step of sampling the original intermediate result corresponding to each sample data of the second device obtained by calculation to obtain a reduced intermediate result corresponding to a part of sample data of the second device includes:
respectively carrying out weighted summation on each sample data of the second equipment by using the model parameters of the second equipment, and calculating to obtain an original intermediate result of the second equipment;
splitting the original intermediate result of the second device based on a threshold value to obtain a first sub-original intermediate result and a second sub-original intermediate result, wherein each piece of data in the first sub-original intermediate result is smaller than or equal to the threshold value, and each piece of data in the second sub-original intermediate result is larger than the threshold value;
grouping all data of the first sub-original intermediate result, determining respective representative data of each group, and forming a third sub-original intermediate result by the representative data of each group;
and obtaining a reduced intermediate result of the second device based on the third sub-original intermediate result and the second sub-original intermediate result.
In order to achieve the above object, the present invention further provides a longitudinal federal learning system optimization device, including: a memory, a processor, and a longitudinal federated learning system optimization program stored on the memory and executable on the processor, the longitudinal federated learning system optimization program when executed by the processor implementing the steps of the longitudinal federated learning system optimization method as described above.
In addition, to achieve the above object, the present invention further provides a readable storage medium, on which a longitudinal federated learning system optimization program is stored, and the longitudinal federated learning system optimization program, when executed by a processor, implements the steps of the longitudinal federated learning system optimization method as described above.
In the invention, an encrypted reduced intermediate result of the second device sent by the second device is received, wherein the second device samples an original intermediate result corresponding to each sample data of the second device obtained by calculation to obtain a reduced intermediate result corresponding to a part of sample data of the second device, encrypts the reduced intermediate result of the second device to obtain an encrypted reduced intermediate result of the second device, performs data padding on the encrypted reduced intermediate result of the second device to obtain an encrypted padded intermediate result of the second device, wherein the data quantity of the encrypted padded intermediate result is the same as the data quantity of the original intermediate result, and then calculates an encrypted first gradient value corresponding to a model parameter in the first device by using the encrypted padded intermediate result of the second device, and updating the model parameters of the first equipment based on the encrypted first gradient value, and circularly iterating until the preset stop condition is met, so as to obtain the trained target model parameters of the first equipment. By reducing the number of data contained in the intermediate result corresponding to the participating equipment, the data quantity needing encryption and communication is reduced, the encryption and communication cost is reduced, and meanwhile, the longitudinal federal modeling time is greatly shortened.
Drawings
FIG. 1 is a schematic diagram of a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of a first embodiment of a longitudinal federated learning system optimization method of the present invention;
fig. 3 is a schematic diagram of sample data according to an embodiment of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
As shown in fig. 1, fig. 1 is a schematic structural diagram of a hardware operating environment according to an embodiment of the present invention.
It should be noted that fig. 1 is a schematic structural diagram of a hardware operating environment of a device that can be optimized for a longitudinal federal learning system. The longitudinal federal learning system optimization device in the embodiment of the invention can be a PC, and can also be a terminal device with a display function, such as a smart phone, a smart television, a tablet personal computer, a portable computer and the like.
As shown in fig. 1, the longitudinal federal learning system optimization device may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, a communication bus 1002. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (e.g., a magnetic disk memory). The memory 1005 may alternatively be a storage device separate from the processor 1001.
Those skilled in the art will appreciate that the system architecture shown in fig. 1 does not constitute a limitation of a terminal system, and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.
As shown in fig. 1, the memory 1005, which is a readable storage medium, may include therein an operating system, a network communication module, a user interface module, and a longitudinal federal learning system optimization program.
In the system shown in fig. 1, the network interface 1004 is mainly used for connecting to a backend server and communicating with the backend server; the user interface 1003 is mainly used for connecting a client (client) and performing data communication with the client; and processor 1001 may be used to invoke a longitudinal federated learning system optimization program stored in memory 1005.
In this embodiment, the terminal system includes: the system comprises a memory 1005, a processor 1001 and a longitudinal federal learning system optimization program stored in the memory 1005 and capable of running on the processor 1001, wherein when the processor 1001 calls the longitudinal federal learning system optimization program stored in the memory 1005, the steps of the longitudinal federal learning system optimization method provided by each embodiment of the application are executed.
Based on the structure, various embodiments of the longitudinal federal learning system optimization method are provided.
While a logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in an order different than presented herein. The first device and the second device related in the embodiment of the present invention may be participating devices participating in federal learning of longitudinal federal learning, and the participating devices may be devices such as a smart phone, a personal computer, and a server.
Referring to fig. 2, fig. 2 is a schematic flow chart of a first embodiment of the longitudinal federated learning system optimization method of the present invention. In this embodiment, the longitudinal federated learning system optimization method includes:
step S10, receiving an encrypted and simplified intermediate result of the second device sent by the second device, where the second device is configured to sample an original intermediate result corresponding to each sample data of the second device obtained through calculation, obtain a simplified intermediate result corresponding to a part of sample data of the second device, and encrypt the simplified intermediate result of the first device, to obtain an encrypted and simplified intermediate result of the first device;
in this embodiment, the first device establishes a communication connection with the second device in advance. The local data of the first device and the second device have an overlapping part in user dimension, and have different parts (possibly completely different) in data characteristics, the first device and the second device perform sample alignment by adopting respective local data to determine common users and different data characteristics of the two devices, the first device takes the data of the common users in the local data as training data, the second device takes the data of the common users in the local data, which is different from the data characteristics of the first device, as training data, that is, the finally determined first sample data and the second sample data are the same in user and different in data characteristics. The sample alignment between the first device and the second device may be performed by using an existing sample alignment technology, which is not described in detail herein. For example, fig. 3 is a schematic diagram of sample data in a first device and a second device, where the local data of the first device includes 3 users { U1, U2, U3}, the data characteristics include { X1, X2, X3}, the local data of the second device includes 3 users { U1, U2, U4}, and the data characteristics include { X4, X5 }. After sample alignment, the training data determined by the first device is the data of users U1 and U2 under data characteristics X1, X2 and X3, and the training data determined by the second device is the data of users U1 and U2 under data characteristics X4 and X5.
Specifically, in a first round of model parameter updating process of longitudinal federated linear model learning, a first device and a second device interact in an encryption mode to calculate intermediate results of a gradient and a loss function, wherein encryption adopts a homomorphic encryption algorithm, a third party coordinator trusted by the first device and the second device jointly generates a public key and a private key, the public key is sent to the first device and the second device for encryption, the first device and the second device send an encrypted gradient value and an encrypted loss function to the coordinator for decryption, and then a local model of the first device and a local model of the second device are updated according to the decrypted gradient value.
The linear models to which the present application relates include, but are not limited to: logistic regression, linear regression, poisson regression, and other linear model algorithms based on weight learning. For convenience of description, the invention takes training of a longitudinal logistic regression model as an example to explain that a logistic regression model is jointly constructed by the first equipment and the second equipment participating in longitudinal federated learning. Second device possession data
Wherein D isAData set representing a second device, the first device possessing data
And a label
Wherein D isBA data set representing a first device is provided,
and
are all multi-dimensional vectors, and yiIs a scalar (e.g., a scalar taking the value 0 or 1, indicating yes or no). Definition of
Wherein, wAAnd wBAre respectively corresponding to
And
machine learning model parameters of
The loss function loss (also called cost function) is:
loss is calculated in the first equipment, and according to the definition of the loss function, the second equipment is required to send the intermediate result uAAnd
to the first device for the first device to calculate the loss value. In the federal training process, the intermediate result needs to be encrypted to avoid data privacy disclosure, so that the second device sends the encrypted intermediate result [ [ u ] ]A]]And
to a first apparatus, wherein [ ]]]Indicating homomorphic encryption.
Further, define
After homomorphic encryption
The gradient function G is that G ═ dx ═ ∑ dxiThen [ [ G ]]]＝[[dx]]＝∑[[d]]xiThe first device receives the encrypted intermediate result [ u ] sent by the second deviceA]]And u of the first deviceBCalculated to obtain [ [ d ]]]According to [ [ d ]]]Further calculating an encrypted gradient value [ G ] of the local model of the first deviceB]]While the first device transmits [ [ d ]]]To the second device for the second device to calculate the encrypted gradient value of the local model [ [ G ]A]]。
In particular uAAnd
respectively with
Is the same, the number of samples is usually very large, and the second device needs to be paired with uAAnd
the encryption is performed and then sent to the first device for interaction, so the whole encryption process is very time consuming and the traffic is very large. Second device to original intermediate result uAAnd
sampling to obtain a simplified intermediate result u'AAnd
the data size from the original intermediate result to the simplified intermediate result is reduced in data dimension, namely the data number in the simplified intermediate result is smaller than that of the original intermediate result, so that the data size needing encryption and communication is reduced, and the encryption and communication cost is reduced. Further, the second device encrypts the simplified intermediate result to obtain an encrypted simplified intermediate result of the second device, and then sends the encrypted simplified intermediate result to the first device. Original intermediate result uAAnd
the process is similar, in this embodiment, with uAThe description is given for the sake of example.
Step S20, performing data padding on the encrypted and simplified intermediate result of the second device to obtain an encrypted and padded intermediate result of the second device, where the data quantity of the encrypted and padded intermediate result is the same as the data quantity of the original intermediate result;
in the present embodiment, it is preferred that,
the intermediate results of the first device and the second device are required to be subjected to data alignment and then to related calculation, so that the first device needs to perform data alignment after receiving the encrypted and reduced intermediate results of the second deviceAnd (4) padding to obtain an encrypted padding intermediate result, and ensuring that the data quantity of the encrypted padding intermediate result is the same as that of the original intermediate result.
Specifically, step S20 includes:
step S21, obtaining a sampling comparison table of the second device, and determining padding data and a padding location corresponding to the padding data in the encrypted and reduced intermediate result of the second device based on the sampling comparison table of the second device;
step S22, inserting the padding data into the encrypted and reduced intermediate result of the second device based on the padding position to obtain an encrypted and supplemented intermediate result of the second device.
In this embodiment, the sampling comparison table is generated when the second device performs sampling processing on the original intermediate result, and the sampling comparison table records the substitution relationship between each data in the reduced intermediate result and the data in the original intermediate result, for example, data a in the reduced intermediate result is substitution data of data 1, data 2, and data 3 in the original intermediate result, and data a can be used to recover data 1, data 2, and data 3. Because the homomorphic encryption algorithm is adopted for the simplified intermediate result, the sequence of data cannot be influenced in the encryption process, so that the data supplementation can be carried out on the encrypted and simplified intermediate result according to the sampling comparison table, and the encrypted and supplemented intermediate result of the second equipment is ensured to be aligned with the corresponding data in the first equipment.
Specifically, a sampling comparison table of the second device is obtained, the sampling comparison table is sent to the first device by the second device, the filling data is selected from the encryption-reduction intermediate result of the second device according to the sampling comparison table, and then it is determined which data is replaced by the filling data, for example, the filling data is data a, which is replaced by data 1, data 2 and data 3, it is to be noted that data 1, data 2 and data 3 are not in the encryption-reduction intermediate result, only the substitution relationship between data a and data 1, data 2 and data 3 is recorded in the sampling comparison table, and in the process of completing the encryption-reduction intermediate result, data a needs to be supplemented to the positions of data 1, data 2 and data 3. And further determining a filling position corresponding to the filling data, and inserting the corresponding filling data into the filling position in the encrypted and simplified intermediate result of the second device to obtain an encrypted and supplemented intermediate result of the second device.
Step S30, calculating an encrypted first gradient value corresponding to the model parameter in the first device by using the encrypted padding intermediate result of the second device, updating the model parameter of the first device based on the encrypted first gradient value, and performing loop iteration until a preset stop condition is met, to obtain a trained target model parameter of the first device.
In this embodiment, after the obtained encryption and supplementation intermediate result of the second device is used, the encrypted first gradient value corresponding to the model parameter in the first device is calculated with the encryption and supplementation intermediate result of the first device, the coordinator that sends the encrypted first gradient value is decrypted, the coordinator sends the decrypted first gradient value back to the first device, and the first device updates the local model parameter by using the first gradient value. And meanwhile, calculating an encrypted loss function by using the encrypted filling intermediate result of the first equipment and the encrypted filling intermediate result of the second equipment, sending a coordinator, decrypting the encrypted loss function by the coordinator, detecting whether a preset stopping condition is met, and continuing the next round of iterative training if the preset stopping condition is not met.
Specifically, step S30 includes:
step S31, calculating to obtain an encrypted and simplified intermediate result of the first device for calculating the gradient value;
specifically, step S31 includes:
step a, the first equipment performs sampling processing on original intermediate results corresponding to all sample data of the first equipment obtained through calculation to obtain simplified intermediate results corresponding to partial sample data of the first equipment;
and b, encrypting the simplified intermediate result of the first equipment to obtain the encrypted simplified intermediate result of the first equipment.
In this embodiment, the original intermediate result of the first device needs to be encrypted before the intermediate result is computed by being completed with the encryption of the second device, so as to reduce the encryption time and increase the model training speed, the original intermediate result of the first device is also sampled, thereby reducing the encrypted data amount and saving the encryption cost and the model training time.
Calculating to obtain original intermediate results corresponding to each sample data of the first equipment according to the model parameters of the first equipment and the data owned by the first equipment,
wherein, wBAre the parameters of the model of the first device,
is data owned by the first device. And sampling the original intermediate result of the first equipment to obtain a simplified intermediate result of the first equipment.
To avoid loss of accuracy in the training of the longitudinal logistic regression model, only u with smaller absolute values is replacediValue of, for larger values of uiThe original value is retained. The specific process of sampling is to split the original intermediate result of the first device according to a threshold to obtain two subsets of the original intermediate result, wherein each data in the first subset is smaller than or equal to the threshold, each data in the second subset is larger than the threshold, the threshold is determined according to actual conditions, and only the data in the first subset is sampled. Grouping the data in the first subset, determining respective representative data of each group, forming a third subset by the representative data of each group, wherein the data of the third subset and the data of the second subset are the simplified intermediate result of the first device, further encrypting the simplified intermediate result of the first device, and the encryption algorithm adopts homomorphic encryption to obtain the encrypted simplified intermediate result of the first device.
Step S32, utilizing the encryption of the second device to fill up the intermediate result and the encryption of the first device to reduce the intermediate result, and calculating to obtain the encryption intermediate result of the first device;
specifically, step S32 includes:
c, performing data completion on the encrypted and simplified intermediate result of the first equipment to obtain an encrypted and completed intermediate result of the first equipment;
and d, calculating to obtain an encrypted intermediate result of the first equipment by using the encrypted filling intermediate result of the first equipment and the encrypted filling intermediate result of the second equipment.
In this embodiment, the data padding is performed on the encrypted and simplified intermediate result of the first device to obtain an encrypted and padded intermediate result of the first device, and the specific process is as follows: the method comprises the steps of obtaining a sampling comparison table of first equipment, determining filling data and filling positions corresponding to the filling data in an encryption and simplification intermediate result of the first equipment according to the sampling comparison table of the first equipment, inserting the filling data in the encryption and simplification intermediate result of the first equipment according to the filling positions to obtain an encryption and completion intermediate result of the first equipment, wherein the data quantity of the encryption and completion intermediate result of the first equipment is the same as that of an original intermediate result of the first equipment.
Further, the intermediate result of encryption and padding of the first device and the intermediate result of encryption and padding of the second device are aligned, and the intermediate result of encryption and padding of the first device [ [ d ] ] is calculated by using the intermediate result of encryption and padding of the first device and the intermediate result of encryption and padding of the second device.
Step S33, calculating an encrypted first gradient value corresponding to the model parameter in the first device using the encrypted intermediate result of the first device.
In this embodiment, the encrypted intermediate result of the first device [ [ d ]]]Encrypting a first gradient value [ G ] corresponding to a model parameter in a first deviceB]]，
And calculating to obtain an encrypted first gradient value according to the encrypted intermediate result of the first device and the data owned by the first device.
The method for optimizing the longitudinal federal learning system provided in this embodiment includes receiving an encrypted and simplified intermediate result of the second device sent by the second device, performing data completion on the encrypted and simplified intermediate result of the second device to obtain an encrypted and completed intermediate result of the second device, calculating an encrypted first gradient value corresponding to a model parameter in the first device by using the encrypted and completed intermediate result of the second device, updating the model parameter of the first device based on the encrypted first gradient value, and performing loop iteration until a preset stop condition is met, so as to obtain a target model parameter of the trained first device. In the longitudinal federal training, the data quantity required to be encrypted and communicated is reduced by reducing the data number contained in the intermediate result of the participating equipment, so that the encryption and communication cost is reduced, and meanwhile, the longitudinal federal modeling time is greatly shortened.
Further, according to the first embodiment, a second embodiment of the method for optimizing a longitudinal federated learning system according to the present invention provides a method for optimizing a longitudinal federated learning system, where the method for optimizing a longitudinal federated learning system is applied to a second device, and the second device may be a smart phone, a personal computer, or other devices, and the method for optimizing a longitudinal federated learning system includes:
step A10, sampling the original intermediate results corresponding to each sample data of the second device obtained by calculation to obtain a simplified intermediate result corresponding to part of sample data of the second device;
in this embodiment, in a first round of model parameter updating process of longitudinal federal learning, a first device and a second device interact in an encryption form to calculate intermediate results of a gradient and a loss function, wherein encryption adopts a homomorphic encryption algorithm, a third party coordinator trusted by the first device and the second device generates a public key and a private key, sends the public key to the first device and the second device for encryption, the first device and the second device send an encrypted gradient value and an encrypted loss function to the coordinator for decryption, and then updates local models of the first device and the second device according to the decrypted gradient value.
The linear models to which the present application relates include, but are not limited to: logistic regression, linear regression, poisson regression, and other linear model algorithms based on weight learning. For convenience of description, the invention adopts a longitudinal logistic regression modelTraining is taken as an example to illustrate that the first equipment and the second equipment which participate in longitudinal federated learning jointly construct a logistic regression model. . Second device possession data
Wherein D isAData set representing a second device, the first device possessing data
And a label
Wherein D isBA data set representing a first device is provided,
and
are all multi-dimensional vectors, and yiIs a scalar (e.g., a scalar taking the value 0 or 1, indicating yes or no). Definition of
Wherein, wAAnd wBAre respectively corresponding to
And
machine learning model parameters of
The loss function loss (also called cost function) is:
loss is calculated at the first device according to a loss functionDefining that the second device is required to send an intermediate result uAAnd
to the first device for the first device to calculate the loss value. In the federal training process, the intermediate result needs to be encrypted to avoid data privacy disclosure, so that the second device sends the encrypted intermediate result [ [ u ] ]A]]And
to a first apparatus, wherein [ ]]]Indicating homomorphic encryption.
Definition of
After homomorphic encryption
The gradient function G is that G ═ dx ═ ∑ dxiThen [ [ G ]]]＝[[dx]]＝∑[[d]]xiThe first device receives the encrypted intermediate result [ u ] sent by the second deviceA]]And u of the first deviceBCalculated to obtain [ [ d ]]]According to [ [ d ]]]Further calculating an encrypted gradient value [ G ] of the local model of the first deviceB]]While the first device transmits [ [ d ]]]To the second device for the second device to calculate the encrypted gradient value of the local model [ [ G ]A]]。
In particular uAAnd
respectively with
Is the same, the number of samples is usually very large, and the second device needs to be paired with uAAnd
the encryption is carried out and then the encrypted data is sent to the first device for interaction, so the whole encryption process is very time-consuming andand the traffic is large. Second device to original intermediate result uAAnd
sampling to obtain a simplified intermediate result u'AAnd
the data size from the original intermediate result to the simplified intermediate result is reduced in data dimension, namely the data number in the simplified intermediate result is smaller than that of the original intermediate result, so that the data size needing encryption and communication is reduced, and the encryption and communication cost is reduced. Further, the second device encrypts the simplified intermediate result to obtain an encrypted simplified intermediate result of the second device, and then sends the encrypted simplified intermediate result to the first device. Original intermediate result uAAnd
the process is similar, in this embodiment, with uAThe description is given for the sake of example.
Specifically, step a10 includes:
step a12, splitting the original intermediate result of the second device based on a threshold to obtain a first sub-original intermediate result and a second sub-original intermediate result, where each data in the first sub-original intermediate result is less than or equal to the threshold, and each data in the second sub-original intermediate result is greater than the threshold;
step A13, grouping all data of the first sub-original intermediate result, determining respective representative data of each group, and forming a third sub-original intermediate result by the representative data of each group;
step a14, obtaining a reduced intermediate result of the second device based on the third sub-original intermediate result and the second sub-original intermediate result.
In this embodiment, in order to avoid the loss of accuracy in the training of the vertical logistic regression model, only u with a small absolute value is replacediValue of, for larger values of uiThe original value is retained. Therefore samplingThe specific processing procedure is to split the original intermediate result of the second device according to the threshold value to obtain two subsets of the original intermediate result: the data in the first sub-original intermediate result is less than or equal to a threshold value, the data in the second sub-original intermediate result is greater than the threshold value, the threshold value is determined according to actual conditions, and only the data in the first sub-original intermediate result is sampled. And grouping the data in the first sub-original intermediate result, determining respective representative data of each group, forming a third sub-original intermediate result by the representative data of each group, wherein the data of the third sub-original intermediate result and the second sub-original intermediate result are the simplified intermediate result of the second device. The specific method for grouping the data in the first sub-original intermediate result and determining the representative data can be determined according to actual conditions, for example, the data in the first sub-original intermediate result are arranged in descending order, and then are averagely divided into N groups, each group calculates the average, and the average is used as the representative data of each group; and N initial clustering centers can be manually set, a final clustering center point is obtained by using k-means, and the final clustering center point is used as representative data of each group.
Step a20, encrypting a simplified intermediate result of a second device to obtain a simplified intermediate result corresponding to a part of sample data of the second device, and sending the simplified intermediate result to the first device, so that the first device feeds back an encrypted intermediate result of the first device based on the encrypted simplified intermediate result of the second device, wherein the first device performs data padding on the received encrypted simplified intermediate result of the second device to obtain an encrypted padded intermediate result of the second device, and calculates the encrypted intermediate result of the first device by using the encrypted padded intermediate result of the second device;
in this embodiment, the simplified intermediate result of the second device is further encrypted, the encryption algorithm uses homomorphic encryption to obtain an encrypted simplified intermediate result of the second device, and the encrypted simplified intermediate result of the second device is sent to the first device. The first device receives the encrypted and reduced intermediate result of the second device, data supplementation is required to be carried out on the encrypted and reduced intermediate result of the second device, the encrypted and supplemented intermediate result of the second device is obtained, then the encrypted and supplemented intermediate result of the first device and the encrypted and supplemented intermediate result of the second device are utilized, the encrypted and supplemented intermediate result of the first device is obtained through calculation, the encrypted and supplemented intermediate result of the first device is sent to the second device through the first device, and the encrypted and supplemented intermediate result of the first device and the encrypted and supplemented intermediate result of the second device are aligned in data.
Step A30, obtaining an encrypted second gradient value corresponding to the model parameter in the second device by utilizing the encrypted intermediate result of the first device, updating the model parameter of the second device based on the encrypted second gradient value, and obtaining the trained target model parameter of the second device after iteration in a circulating manner until a preset stop condition is met.
In this embodiment, the resulting encrypted intermediate result of the first device [ [ d ]]]Encrypting the first gradient value [ G ] corresponding to the model parameter in the second deviceA]]，
And settling to obtain the encrypted second gradient value according to the encrypted intermediate result of the first device and the owned data of the second device. And decrypting the coordinator sent by the encrypted second gradient value, sending the decrypted second gradient value back to the second equipment by the coordinator, and updating the local model parameter by the second equipment by using the second gradient value. Meanwhile, the coordinator detects whether a preset stop condition is met, and if the preset stop condition is not met, the next round of iterative training is continued.
The method for optimizing a longitudinal federated learning system provided in this embodiment includes sampling an original intermediate result corresponding to each sample data of a second device obtained through calculation to obtain a reduced intermediate result corresponding to a part of sample data of the second device, encrypting the reduced intermediate result of the second device to obtain a reduced intermediate result corresponding to a part of sample data of the second device, sending the reduced intermediate result to the first device, so that the first device feeds back an encrypted intermediate result of the first device based on the encrypted reduced intermediate result of the second device, obtaining an encrypted second gradient value corresponding to a model parameter in the second device through calculation using the encrypted intermediate result of the first device, updating the model parameter of the second device based on the encrypted second gradient value, and performing loop iteration until a preset stop condition is detected to be satisfied, and obtaining the target model parameters of the trained second equipment. In the longitudinal federal training, the data quantity required to be encrypted and communicated is reduced by reducing the data number contained in the intermediate result of the participating equipment, so that the encryption and communication cost is reduced, and meanwhile, the longitudinal federal modeling time is greatly shortened.
In addition, an embodiment of the present invention further provides a readable storage medium, where a longitudinal federated learning system optimization program is stored on the storage medium, and when being executed by a processor, the longitudinal federated learning system optimization program implements the steps of the longitudinal federated learning system optimization method described below.
The embodiments of the longitudinal federated learning system optimization device and the readable storage medium of the present invention may refer to the embodiments of the longitudinal federated learning system optimization method of the present invention, and are not described herein again.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. The longitudinal federal learning system optimization method is applied to first equipment participating in longitudinal federal learning, and the first equipment is in communication connection with second equipment, and the longitudinal federal learning system optimization method comprises the following steps:
receiving an encrypted and simplified intermediate result of the second device sent by the second device, wherein the second device is configured to sample an original intermediate result corresponding to each sample data of the second device obtained through calculation to obtain a simplified intermediate result corresponding to a part of sample data of the second device, and encrypt the simplified intermediate result of the second device to obtain an encrypted and simplified intermediate result of the second device;
performing data supplementation on the encrypted and simplified intermediate result of the second device to obtain an encrypted and supplemented intermediate result of the second device, wherein the data quantity of the encrypted and supplemented intermediate result is the same as that of the original intermediate result;
and calculating to obtain an encrypted first gradient value corresponding to the model parameter in the first equipment by using the encrypted filling intermediate result of the second equipment, updating the model parameter of the first equipment based on the encrypted first gradient value, and performing iteration circularly until a preset stopping condition is met to obtain a target model parameter of the trained first equipment.
2. The method for optimizing a longitudinal federated learning system as set forth in claim 1, wherein the step of data-supplementing the cryptographically reduced intermediate results of the second device to obtain cryptographically supplemented intermediate results comprises:
acquiring a sampling comparison table of the second device, and determining filling data and a filling position corresponding to the filling data in an encrypted and simplified intermediate result of the second device based on the sampling comparison table of the second device;
and inserting the filling data into the encrypted and simplified intermediate result of the second device based on the filling position to obtain an encrypted and supplemented intermediate result of the second device.
3. The method for optimizing a longitudinal federal learning system as claimed in any one of claims 1 to 2, wherein the step of calculating the encrypted first gradient value corresponding to the model parameter in the first device using the encrypted padding intermediate result of the second device comprises:
calculating to obtain an encrypted and simplified intermediate result of first equipment for calculating the gradient value;
calculating to obtain an encrypted intermediate result of the first device by utilizing the encrypted filling intermediate result of the second device and the encrypted simplified intermediate result of the first device;
and calculating to obtain an encrypted first gradient value corresponding to the model parameter in the first equipment by using the encrypted intermediate result of the first equipment.
4. The method for longitudinal federal learning system optimization as claimed in claim 3, wherein said step of calculating a cryptographically condensed intermediate result of the first device used to calculate the gradient values comprises:
the first equipment performs sampling processing on the original intermediate result corresponding to each sample data of the first equipment obtained through calculation to obtain a simplified intermediate result corresponding to partial sample data of the first equipment;
and encrypting the simplified intermediate result of the first device to obtain an encrypted simplified intermediate result of the first device.
5. The method for optimizing a longitudinal federated learning system as defined in claim 3, wherein the step of calculating the intermediate result of encryption for the first device using the intermediate result of encryption padding for the second device and the intermediate result of encryption compaction for the first device comprises:
performing data complementation on the encrypted and simplified intermediate result of the first device to obtain an encrypted and complemented intermediate result of the first device;
and calculating to obtain an encrypted intermediate result of the first device by using the encrypted filling intermediate result of the first device and the encrypted filling intermediate result of the second device.
6. A longitudinal federated learning system optimization method is applied to second equipment participating in longitudinal federated learning and comprises the following steps:
sampling the original intermediate result corresponding to each sample data of the second equipment obtained by calculation to obtain a simplified intermediate result corresponding to part of sample data of the second equipment;
encrypting a simplified intermediate result of second equipment to obtain a simplified intermediate result corresponding to part of sample data of the second equipment, and sending the simplified intermediate result to the first equipment, so that the first equipment feeds back the encrypted intermediate result of the first equipment based on the encrypted simplified intermediate result of the second equipment, wherein the first equipment is used for performing data complementation on the received encrypted simplified intermediate result of the second equipment to obtain an encrypted complemented intermediate result of the second equipment, and calculating by using the encrypted complemented intermediate result of the second equipment to obtain the encrypted intermediate result of the first equipment;
and calculating to obtain an encrypted second gradient value corresponding to the model parameter in the second equipment by using the encrypted intermediate result of the first equipment, updating the model parameter of the second equipment based on the encrypted second gradient value, and performing loop iteration until a preset stop condition is met to obtain a target model parameter of the trained second equipment.
7. The method for optimizing a longitudinal federated learning system according to claim 6, wherein the step of sampling the original intermediate results corresponding to the various sample data of the second device obtained by calculation to obtain the reduced intermediate results corresponding to a part of sample data of the second device includes:
splitting the original intermediate result of the second device based on a threshold value to obtain a first sub-original intermediate result and a second sub-original intermediate result, wherein each piece of data in the first sub-original intermediate result is smaller than or equal to the threshold value, and each piece of data in the second sub-original intermediate result is larger than the threshold value;
grouping all data of the first sub-original intermediate result, determining respective representative data of each group, and forming a third sub-original intermediate result by the representative data of each group;
and obtaining a reduced intermediate result of the second device based on the third sub-original intermediate result and the second sub-original intermediate result.
8. A longitudinal federated learning system optimization apparatus, comprising: a memory, a processor, and a longitudinal federated learning system optimization program stored on the memory and executable on the processor, the longitudinal federated learning system optimization program when executed by the processor implementing the steps of the longitudinal federated learning system optimization method of any one of claims 1 to 5.
9. A longitudinal federated learning system optimization apparatus, comprising: a memory, a processor, and a longitudinal federated learning system optimization program stored on the memory and executable on the processor, the longitudinal federated learning system optimization program when executed by the processor implementing the steps of the longitudinal federated learning system optimization method of any one of claims 6 to 7.
10. A readable storage medium having stored thereon a longitudinal federated learning system optimization program that, when executed by a processor, performs the steps of the longitudinal federated learning system optimization method of any of claims 1-7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010089045.6A CN111340247B (en) | 2020-02-12 | 2020-02-12 | Longitudinal federal learning system optimization method, device and readable storage medium |
| PCT/CN2020/129255 WO2021159798A1 (en) | 2020-02-12 | 2020-11-17 | Method for optimizing longitudinal federated learning system, device and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010089045.6A CN111340247B (en) | 2020-02-12 | 2020-02-12 | Longitudinal federal learning system optimization method, device and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111340247A true CN111340247A (en) | 2020-06-26 |
| CN111340247B CN111340247B (en) | 2024-10-15 |
Family
ID=71183882
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010089045.6A Active CN111340247B (en) | 2020-02-12 | 2020-02-12 | Longitudinal federal learning system optimization method, device and readable storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN111340247B (en) |
| WO (1) | WO2021159798A1 (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112700010A (en) * | 2020-12-30 | 2021-04-23 | 深圳前海微众银行股份有限公司 | Feature completion method, device, equipment and storage medium based on federal learning |
| CN112949866A (en) * | 2021-03-25 | 2021-06-11 | 深圳前海微众银行股份有限公司 | Poisson regression model training method and device, electronic equipment and storage medium |
| CN113011603A (en) * | 2021-03-17 | 2021-06-22 | 深圳前海微众银行股份有限公司 | Model parameter updating method, device, equipment, storage medium and program product |
| CN113240127A (en) * | 2021-04-07 | 2021-08-10 | 睿蜂群(北京)科技有限公司 | Federal learning-based training method and device, electronic equipment and storage medium |
| WO2021159798A1 (en) * | 2020-02-12 | 2021-08-19 | 深圳前海微众银行股份有限公司 | Method for optimizing longitudinal federated learning system, device and readable storage medium |
| CN113704776A (en) * | 2021-07-15 | 2021-11-26 | 杭州医康慧联科技股份有限公司 | Machine learning method based on federal learning |
| CN113762531A (en) * | 2021-03-19 | 2021-12-07 | 北京沃东天骏信息技术有限公司 | Federated learning system |
| CN113806759A (en) * | 2020-12-28 | 2021-12-17 | 京东科技控股股份有限公司 | Federal learning model training method and device, electronic equipment and storage medium |
| CN114077901A (en) * | 2021-11-23 | 2022-02-22 | 山东大学 | User position prediction framework based on clustering and used for image federation learning |
| CN114140144A (en) * | 2020-09-04 | 2022-03-04 | 京东科技控股股份有限公司 | Information delivery method and device, and information delivery decision model acquisition method and device |
| CN114330759A (en) * | 2022-03-08 | 2022-04-12 | 富算科技(上海)有限公司 | Training method and system for longitudinal federated learning model |
| WO2022142366A1 (en) * | 2020-12-31 | 2022-07-07 | 华为技术有限公司 | Method and apparatus for updating machine learning model |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101375284A (en) * | 2004-10-25 | 2009-02-25 | 里克·L·奥尔西尼 | Secure data parser method and system |
| US20120198241A1 (en) * | 2011-01-27 | 2012-08-02 | Security First Corp. | Systems and methods for securing data |
| US20160344428A1 (en) * | 2014-01-30 | 2016-11-24 | Hewlett Packard Enterprise Development Lp | Joint encryption and error correction encoding |
| CN110288094A (en) * | 2019-06-10 | 2019-09-27 | 深圳前海微众银行股份有限公司 | Model parameter training method and device based on federation's study |
| CN110633806A (en) * | 2019-10-21 | 2019-12-31 | 深圳前海微众银行股份有限公司 | Longitudinal federated learning system optimization method, device, equipment and readable storage medium |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109492420B (en) * | 2018-12-28 | 2021-07-20 | 深圳前海微众银行股份有限公司 | Model parameter training method, terminal, system and medium based on federal learning |
| CN110245510B (en) * | 2019-06-19 | 2021-12-07 | 北京百度网讯科技有限公司 | Method and apparatus for predicting information |
| CN110428058B (en) * | 2019-08-08 | 2024-04-26 | 深圳前海微众银行股份有限公司 | Federal learning model training method, device, terminal equipment and storage medium |
| KR20190103088A (en) * | 2019-08-15 | 2019-09-04 | 엘지전자 주식회사 | Method and apparatus for recognizing a business card using federated learning |
| CN111340247B (en) * | 2020-02-12 | 2024-10-15 | 深圳前海微众银行股份有限公司 | Longitudinal federal learning system optimization method, device and readable storage medium |
-
2020
- 2020-02-12 CN CN202010089045.6A patent/CN111340247B/en active Active
- 2020-11-17 WO PCT/CN2020/129255 patent/WO2021159798A1/en active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101375284A (en) * | 2004-10-25 | 2009-02-25 | 里克·L·奥尔西尼 | Secure data parser method and system |
| US20120198241A1 (en) * | 2011-01-27 | 2012-08-02 | Security First Corp. | Systems and methods for securing data |
| US20160344428A1 (en) * | 2014-01-30 | 2016-11-24 | Hewlett Packard Enterprise Development Lp | Joint encryption and error correction encoding |
| CN110288094A (en) * | 2019-06-10 | 2019-09-27 | 深圳前海微众银行股份有限公司 | Model parameter training method and device based on federation's study |
| CN110633806A (en) * | 2019-10-21 | 2019-12-31 | 深圳前海微众银行股份有限公司 | Longitudinal federated learning system optimization method, device, equipment and readable storage medium |
Non-Patent Citations (2)
| Title |
|---|
| FARZIN HADDADPOUR ET AL.,: "On the Convergence of Local Descent Methods in Federated Learning", 《ARXIV》, 31 October 2019 (2019-10-31), pages 1 - 49 * |
| 魏雅婷 等,: "联邦可视化：一种隐私保护的可视化新模型", 《智能科学与技术学报》, vol. 1, no. 4, 31 December 2019 (2019-12-31), pages 415 - 420 * |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021159798A1 (en) * | 2020-02-12 | 2021-08-19 | 深圳前海微众银行股份有限公司 | Method for optimizing longitudinal federated learning system, device and readable storage medium |
| CN114140144A (en) * | 2020-09-04 | 2022-03-04 | 京东科技控股股份有限公司 | Information delivery method and device, and information delivery decision model acquisition method and device |
| CN113806759A (en) * | 2020-12-28 | 2021-12-17 | 京东科技控股股份有限公司 | Federal learning model training method and device, electronic equipment and storage medium |
| CN112700010A (en) * | 2020-12-30 | 2021-04-23 | 深圳前海微众银行股份有限公司 | Feature completion method, device, equipment and storage medium based on federal learning |
| WO2022142366A1 (en) * | 2020-12-31 | 2022-07-07 | 华为技术有限公司 | Method and apparatus for updating machine learning model |
| CN113011603A (en) * | 2021-03-17 | 2021-06-22 | 深圳前海微众银行股份有限公司 | Model parameter updating method, device, equipment, storage medium and program product |
| CN113762531A (en) * | 2021-03-19 | 2021-12-07 | 北京沃东天骏信息技术有限公司 | Federated learning system |
| CN112949866A (en) * | 2021-03-25 | 2021-06-11 | 深圳前海微众银行股份有限公司 | Poisson regression model training method and device, electronic equipment and storage medium |
| CN113240127A (en) * | 2021-04-07 | 2021-08-10 | 睿蜂群(北京)科技有限公司 | Federal learning-based training method and device, electronic equipment and storage medium |
| CN113704776A (en) * | 2021-07-15 | 2021-11-26 | 杭州医康慧联科技股份有限公司 | Machine learning method based on federal learning |
| CN114077901A (en) * | 2021-11-23 | 2022-02-22 | 山东大学 | User position prediction framework based on clustering and used for image federation learning |
| CN114077901B (en) * | 2021-11-23 | 2024-05-24 | 山东大学 | User position prediction method based on clustering graph federation learning |
| CN114330759A (en) * | 2022-03-08 | 2022-04-12 | 富算科技(上海)有限公司 | Training method and system for longitudinal federated learning model |
| CN114330759B (en) * | 2022-03-08 | 2022-08-02 | 富算科技(上海)有限公司 | Training method and system for longitudinal federated learning model |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2021159798A1 (en) | 2021-08-19 |
| CN111340247B (en) | 2024-10-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111340247A (en) | Longitudinal federated learning system optimization method, device and readable storage medium | |
| CN109165725B (en) | Neural network federal modeling method, equipment and storage medium based on transfer learning | |
| CN111428881B (en) | Recognition model training method, device, equipment and readable storage medium | |
| CN109255444B (en) | Federal modeling method and device based on transfer learning and readable storage medium | |
| CN113239404B (en) | Federal learning method based on differential privacy and chaotic encryption | |
| CN110633806A (en) | Longitudinal federated learning system optimization method, device, equipment and readable storage medium | |
| CN109886417A (en) | Model parameter training method, device, equipment and medium based on federation's study | |
| CN112347500B (en) | Machine learning method, device, system, equipment and storage medium of distributed system | |
| CN110851786A (en) | Longitudinal federated learning optimization method, device, equipment and storage medium | |
| CN111325352B (en) | Model updating method, device, equipment and medium based on longitudinal federal learning | |
| CN111767411B (en) | Knowledge graph representation learning optimization method, device and readable storage medium | |
| CN114691167A (en) | Method and device for updating machine learning model | |
| CN111340453A (en) | Federal learning development method, device, equipment and storage medium | |
| CN110807528A (en) | Feature correlation calculation method, device and computer-readable storage medium | |
| CN113935050B (en) | Feature extraction method and device based on federal learning, electronic equipment and medium | |
| CN116502732B (en) | Federal learning method and system based on trusted execution environment | |
| CN112231309B (en) | Method, device, terminal equipment and medium for removing duplicate of longitudinal federal data statistics | |
| CN114696990A (en) | Multi-party computing method, system and related equipment based on fully homomorphic encryption | |
| CN111414636A (en) | Method, device and equipment for updating recognition model and storage medium | |
| CN112948883B (en) | Method, device and system for multiparty joint modeling of privacy data protection | |
| CN112949866B (en) | Training method and device of poisson regression model, electronic equipment and storage medium | |
| CN111368314A (en) | Modeling and predicting method, device, equipment and storage medium based on cross features | |
| CN114168295A (en) | Hybrid architecture system and task scheduling method based on historical task effect | |
| CN116506227B (en) | Data processing method, device, computer equipment and storage medium | |
| CN115361196A (en) | Service interaction method based on block chain network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |