CN113947211A - Federal learning model training method and device, electronic equipment and storage medium - Google Patents

Federal learning model training method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN113947211A
CN113947211A CN202111183940.5A CN202111183940A CN113947211A CN 113947211 A CN113947211 A CN 113947211A CN 202111183940 A CN202111183940 A CN 202111183940A CN 113947211 A CN113947211 A CN 113947211A
Authority
CN
China
Prior art keywords
training
target
gradient
server
feature
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111183940.5A
Other languages
Chinese (zh)
Inventor
杨恺
王虎
黄志翔
彭南博
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jingdong Technology Holding Co Ltd
Original Assignee
Jingdong Technology Holding Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jingdong Technology Holding Co Ltd filed Critical Jingdong Technology Holding Co Ltd
Priority to CN202111183940.5A priority Critical patent/CN113947211A/en
Publication of CN113947211A publication Critical patent/CN113947211A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/04—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks
    • H04L63/0428—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks wherein the data content is protected, e.g. by encrypting or encapsulating the payload
    • H04L63/0442—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks wherein the data content is protected, e.g. by encrypting or encapsulating the payload wherein the sending and receiving network entities apply asymmetric encryption, i.e. different keys for encryption and decryption
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/008—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols involving homomorphic encryption

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Software Systems (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Electrically Operated Instructional Devices (AREA)

Abstract

The application provides a method and a device for training a federated learning model, electronic equipment and a storage medium, wherein the training method comprises the following steps: sample alignment with a data provider server; respectively numbering the characteristics of the server of the business party and the server of the data provider according to the characteristic quantity of the server of the business party and the server of the data provider to generate a characteristic coding set, and sending the characteristic number and the public key of the server of the data provider to the server of the data provider; acquiring a current sample set and a training parameter set of a federated learning model; performing iterative training on the federated learning model for M times according to the current sample set, the training parameter set and the feature coding set; and acquiring target parameters of the federated learning model obtained by the Mth iterative training. Therefore, the modeling complexity can be reduced while the modeling effect is ensured, so that the joint training between the server of the business party and the server of the data provider is more efficient, and the modeling efficiency is improved.

Description

Federal learning model training method and device, electronic equipment and storage medium
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a method and an apparatus for training a federated learning model, an electronic device, and a storage medium.
Background
With the development of machine learning, more and more machine learning techniques are applied to various industries. The quantity and quality of the data often determine the upper limit of the effectiveness of the machine learning model. However, as regulations and regulations become more stringent and people pay more attention to data security and privacy protection, data islanding is formed. Under the scene, federal learning comes by the fortune, and the joint training can be carried out on the basis that the participators do not share data, so that the problem of data island is solved.
In the related art, federal learning is an encrypted distributed machine learning technology, and various technologies such as information encryption, distributed computation, machine learning and the like are fused. Federal learning can be classified into horizontal federal learning, vertical federal learning, and federal migratory learning according to the characteristics of data held by participants. Under the wind control scene, the application of longitudinal federal learning is wider.
Disclosure of Invention
The embodiment of the first aspect of the application provides a method for training a federated learning model, which can reduce the complexity of modeling while ensuring the modeling effect, so that the federated training between a business side server and a data provider server is more efficient, and the modeling efficiency is further improved.
The embodiment of the second aspect of the application provides a method for training a federated learning model.
The embodiment of the third aspect of the application provides a training device for a federated learning model.
The embodiment of the fourth aspect of the application provides a training device for a joint learning model.
The embodiment of the fifth aspect of the present application provides an electronic device.
A sixth aspect of the present application provides a computer-readable storage medium.
An embodiment of a first aspect of the present application provides a method for training a bang learning model, including:
sample alignment with a data provider server;
respectively acquiring the feature quantity of a business side server and the feature quantity of a data provider server, respectively numbering the features of the business side server and the data provider server according to the feature quantity to generate a feature code set, and sending the feature number and a public key of the data provider server to the data provider server;
acquiring a current sample set and a training parameter set of a federated learning model;
performing iterative training on the federated learning model for M times according to the current sample set, the training parameter set and the feature coding set, wherein M is a positive integer greater than 1; and
and obtaining target parameters of the federated learning model obtained by the Mth iterative training.
According to the method for training the federated learning model, firstly, sample alignment is carried out on a data provider server, then the characteristic numbers of a business side server and the data provider server are respectively obtained, the characteristics of the business side server and the data provider server are respectively numbered according to the characteristic numbers to generate a characteristic code set, the characteristic numbers and public keys of the data provider server are sent to the data provider server, then a current sample set and a training parameter set of the federated learning model are obtained, M times of iterative training are carried out on the federated learning model according to the current sample set, the training parameter set and the characteristic code set, and finally target parameters of the federated learning model obtained through M times of iterative training are obtained. Therefore, the modeling complexity can be reduced while the modeling effect is ensured, so that the joint training between the server of the business party and the server of the data provider is more efficient, and the modeling efficiency is improved.
In addition, the method for training the federal learning model according to the above embodiment of the present application may further have the following additional technical features:
in one embodiment of the present application, the training parameter set includes a feature sampling rate, a training sample upper limit value, a training sample lower limit value, a decision tree number upper limit value, a decision tree number lower limit value, a first parameter change speed, and a second parameter change speed.
In an embodiment of the present application, the training at each iteration includes:
taking the current iterative training in the M iterative training as the Nth iterative training, wherein N is a positive integer smaller than M;
generating a sample sampling rate according to the M, the N, the training sample upper limit value, the training sample lower limit value and the first parameter change speed;
generating a target number according to the M, the N, the upper limit value of the number of decision trees, the lower limit value of the number of decision trees and the second parameter change speed;
selecting samples of the sample sampling rate from the current sample set to generate a target training set;
selecting feature codes of the feature sampling rate from the feature code set to generate a target feature code set;
sending the number of each sample in the target training set and the target feature number of the data provider server in the target feature coding set to the data provider server;
generating target parameters of the federated learning model according to the target training set, the target feature coding set and the target number;
and generating an optimization label of the current sample based on a gradient lifting algorithm and according to the target parameter and the federal learning model, wherein the optimization label is a training label of the current sample of the next round of iterative training.
In an embodiment of the present application, the generating target parameters of the federal learning model according to the target training set, the target feature coding set, and the target number of plants includes:
calculating gradient information of the samples in the target training set, and sending the gradient information to the data provider server;
receiving gradient return information provided by the data provider server;
generating a target split point number according to the gradient return information and the target feature coding set, generating a ciphertext based on a private key and the target split point number, and sending the ciphertext to the data provider server;
receiving a decryption operation value sent by the data provider server, and splitting nodes according to the decryption operation value;
repeating the steps until the model converges to establish a decision tree of the target number, finishing the training of the federated learning model, and obtaining the target parameters through the federated learning model which finishes the training.
In an embodiment of the present application, the calculating gradient information of the samples in the target training set includes:
generating a first-order gradient value and a second-order gradient value of the samples in the target training set;
homomorphically encrypting the first-order gradient value and the second-order gradient value to generate the gradient information.
In an embodiment of the present application, the generating a target split point number according to the gradient return information and the target feature encoding set includes:
respectively generating a plurality of corresponding information gains according to the gradient return information and the target feature coding set;
and selecting the maximum information gain from the plurality of information gains, and taking the number corresponding to the maximum information gain as the target split point number.
In one embodiment of the present application, the performing node splitting according to the decryption operation value includes:
generating split space information according to the decryption operation value;
and splitting nodes according to the samples in the target training set and the splitting space information.
An embodiment of a second aspect of the present application provides a method for training a bang learning model, including:
performing sample alignment with a service side server;
receiving the characteristic number and the public key of the data provider server sent by the service party server;
receiving the number of each sample in a target training set sent by the service side server and the target feature number of the data provider server in a target feature coding set;
receiving gradient information of a currently trained sample sent by the service side server, and acquiring gradient return information according to the gradient information;
sending the gradient return information to the service side server;
receiving a cipher text which is sent by the service side server and generated based on a private key and a target split point number, wherein the target split point number is generated according to the gradient return information and the target feature coding set; and
and decrypting the ciphertext based on the public key to obtain a decrypted operation value, and sending the decrypted operation value to the service side server.
According to the method for training the federated learning model, firstly, sample alignment is carried out on a server of a business party, then a feature number and a public key of a data provider server sent by the server of the business party are received, the number of each sample in a target training set sent by the server of the business party and a target feature number of the data provider server in a target feature coding set are received, gradient information of a currently trained sample sent by the server of the business party is received, gradient return information is obtained according to the gradient information, then the gradient return information is sent to the server of the business party, a ciphertext generated by the server of the business party based on a private key and a target split point number is received, finally the ciphertext is decrypted based on the public key, a decrypted operation value is obtained, and the decrypted operation value is sent to the server of the business party. Therefore, the modeling complexity can be reduced while the modeling effect is ensured, so that the joint training between the server of the business party and the server of the data provider is more efficient, and the modeling efficiency is improved.
In addition, the training device of the federal learning model according to the above embodiment of the present application may have the following additional technical features:
in an embodiment of the present application, the obtaining gradient return information according to the gradient information includes:
determining a feature set according to the target feature number and the feature number of the data provider server;
splitting a sample space according to a splitting threshold value corresponding to each feature in the feature set to obtain a splitting space on a designated side;
acquiring gradient summation information of the splitting space of the designated side corresponding to each feature according to the gradient information, and numbering the gradient summation information;
and generating the gradient return information by using the gradient summation information and the serial number of the gradient summation information.
In an embodiment of the present application, after the numbering the gradient summation information, the method further includes:
and generating the number and a mapping relation among the feature corresponding to the number, the splitting threshold and the gradient summation information corresponding to the number.
An embodiment of a third aspect of the present application provides a training apparatus for a bang learning model, including:
the alignment module is used for aligning samples with the data provider server;
the sending module is used for respectively acquiring the feature quantities of the business side server and the data provider side server, respectively numbering the features of the business side server and the data provider side server according to the feature quantities to generate a feature coding set, and sending the feature number and the public key of the data provider side server to the data provider side server;
the first acquisition module is used for acquiring a current sample set and a training parameter set of the federated learning model;
the iterative training module is used for carrying out M times of iterative training on the federated learning model according to the current sample set, the training parameter set and the feature coding set, wherein M is a positive integer greater than 1; and
and the second acquisition module is used for acquiring the target parameters of the federated learning model obtained by the Mth iterative training.
The training device of the federal learning model in the embodiment of the application firstly carries out sample alignment with the data provider server through the alignment module, then the characteristic quantities of the server of the business party and the server of the data provider are respectively obtained through a sending module, and numbering the characteristics of the server of the service provider and the server of the data provider according to the characteristic quantity respectively to generate a characteristic code set, and the characteristic number and the public key of the data provider server are sent to the data provider server, then the current sample set and the training parameter set of the federated learning model are obtained through the first obtaining module, and the iterative training module is used for obtaining the code set according to the current sample set, the training parameter set and the characteristic code set, and performing iterative training on the federated learning model for M times, and finally obtaining target parameters of the federated learning model obtained by the iterative training for the M time through a second obtaining module. Therefore, the modeling complexity can be reduced while the modeling effect is ensured, so that the joint training between the server of the business party and the server of the data provider is more efficient, and the modeling efficiency is improved.
In addition, the training device of the federal learning model according to the above embodiment of the present application may have the following additional technical features:
in one embodiment of the present application, the training parameter set includes a feature sampling rate, a training sample upper limit value, a training sample lower limit value, a decision tree number upper limit value, a decision tree number lower limit value, a first parameter change speed, and a second parameter change speed.
In one embodiment of the present application, the iterative training module includes:
a setting submodule, configured to use a current iterative training in the M iterative training times as an nth iterative training time, where N is a positive integer smaller than M;
the first generation submodule is used for generating a sample sampling rate according to the M, the N, the upper limit value of the training sample, the lower limit value of the training sample and the first parameter change speed;
a second generation submodule, configured to generate a target number according to the M, the N, the upper limit of the number of decision trees, the lower limit of the number of decision trees, and the second parameter change speed;
the third generation submodule is used for selecting samples of the sample sampling rate from the current sample set so as to generate a target training set;
the fourth generation submodule is used for selecting the feature codes of the feature sampling rate from the feature code set to generate a target feature code set;
a sending submodule, configured to send the number of each sample in the target training set and the target feature number of the data provider server in the target feature coding set to the data provider server;
a fifth generation submodule, configured to generate a target parameter of the federated learning model according to the target training set, the target feature coding set, and the target number;
and a sixth generation submodule, configured to generate an optimization label of the current sample according to the target parameter and the federal learning model based on a gradient boost algorithm, where the optimization label is a training label of the current sample for a next round of iterative training.
In an embodiment of the application, the fifth generation submodule includes:
the calculation unit is used for calculating the gradient information of the samples in the target training set and sending the gradient information to the data provider server;
the receiving unit is used for receiving the gradient return information provided by the data provider server;
the generating unit is used for generating a target split point number according to the gradient return information and the target feature coding set, generating a ciphertext based on a private key and the target split point number, and sending the ciphertext to the data provider server;
the node splitting unit is used for receiving the decryption operation value sent by the data provider server and splitting nodes according to the decryption operation value;
and the obtaining unit is used for repeating the steps until the model converges to establish a decision tree of the target number, finish the training of the federated learning model and obtain the target parameters through the federated learning model which finishes the training.
In an embodiment of the present application, the computing unit is specifically configured to:
generating a first-order gradient value and a second-order gradient value of the samples in the target training set;
homomorphically encrypting the first-order gradient value and the second-order gradient value to generate the gradient information.
In an embodiment of the application, the gradient return information includes a plurality of gradient return information, and each gradient return information corresponds to a corresponding number, where the generating unit is specifically configured to:
respectively generating a plurality of corresponding information gains according to the gradient return information and the target feature coding set;
and selecting the maximum information gain from the plurality of information gains, and taking the number corresponding to the maximum information gain as the target split point number.
In an embodiment of the present application, the node splitting unit is specifically configured to:
generating split space information according to the decryption operation value;
and splitting nodes according to the samples in the target training set and the splitting space information.
An embodiment of a fourth aspect of the present application provides a training device for a bang learning model, including:
the alignment module is used for aligning samples with the service side server;
the first receiving module is used for receiving the characteristic number and the public key of the data provider server sent by the service party server;
the second receiving module is used for receiving the serial number of each sample in the target training set sent by the server of the service party and the target characteristic serial number of the server of the data provider in the target characteristic coding set;
the third receiving module is used for receiving the gradient information of the currently trained sample sent by the service side server and acquiring gradient return information according to the gradient information;
the first sending module is used for sending the gradient return information to the service side server;
a fourth receiving module, configured to receive a ciphertext generated based on a private key and a target split point number, where the target split point number is generated according to the gradient return information and the target feature encoding set, and the ciphertext is sent by the server of the service party; and
and the second sending module is used for decrypting the ciphertext based on the public key to obtain a decrypted operation value and sending the decrypted operation value to the service side server.
The training device of the federal learning model in the embodiment of the application firstly aligns samples with a server of a business party through an aligning module, then receives a feature number and a public key of a data provider server sent by the server of the business party through a first receiving module, receives the number of each sample in a target training set sent by the server of the business party and a target feature number of the server of the data provider in a target feature coding set through a second receiving module, receives gradient information of the currently trained sample sent by the server of the business party through a third receiving module, acquires gradient return information according to the gradient information, then sends the gradient return information to the server of the business party through the first sending module, receives a ciphertext which is sent by the server of the business party and is generated based on a private key and a target split point number through a fourth receiving module, and finally decrypts the ciphertext based on the public key through the second sending module, and obtaining the decryption operation value and sending the decryption operation value to the service side server. Therefore, the modeling complexity can be reduced while the modeling effect is ensured, so that the joint training between the server of the business party and the server of the data provider is more efficient, and the modeling efficiency is improved.
In addition, the training device of the federal learning model according to the above embodiment of the present application may have the following additional technical features:
in an embodiment of the application, the third receiving module is specifically configured to:
determining a feature set according to the target feature number and the feature number of the data provider server;
splitting a sample space according to a splitting threshold value corresponding to each feature in the feature set to obtain a splitting space on a designated side;
acquiring gradient summation information of the splitting space of the designated side corresponding to each feature according to the gradient information, and numbering the gradient summation information;
and generating the gradient return information by using the gradient summation information and the serial number of the gradient summation information.
In an embodiment of the application, the third receiving module is further configured to:
and generating the number and a mapping relation among the feature corresponding to the number, the splitting threshold and the gradient summation information corresponding to the number.
An embodiment of a fifth aspect of the present application provides an electronic device, including: a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method for training the federal learning model as described in the foregoing embodiments of the first aspect or the second aspect when executing the program.
According to the electronic equipment, the processor executes the computer program stored on the memory, the modeling effect can be guaranteed, and meanwhile the modeling complexity can be reduced, so that the joint training between the server of the business party and the server of the data provider is more efficient, and the modeling efficiency is improved.
An embodiment of a sixth aspect of the present application provides a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements a method for training a federal learning model as defined in an embodiment of the first aspect or an embodiment of the second aspect.
The computer-readable storage medium of the embodiment of the application can reduce the complexity of modeling while ensuring the modeling effect by storing the computer program and being executed by the processor, so that the joint training between the server of the business party and the server of the data provider is more efficient, and the modeling efficiency is further improved.
Additional aspects and advantages of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present application.
Drawings
The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 is a schematic flow chart diagram of a method for training a federated learning model in accordance with one embodiment of the present application;
FIG. 2 is a schematic flow chart diagram of a method for training a federated learning model in accordance with another embodiment of the present application;
FIG. 3 is a schematic flow chart diagram of a method for training a federated learning model in accordance with another embodiment of the present application;
FIG. 4 is a schematic flow chart diagram illustrating a method for training a federated learning model in accordance with another embodiment of the present application;
FIG. 5 is a schematic diagram of a method of training a federated learning model in accordance with an embodiment of the present application;
FIG. 6 is a schematic flow chart diagram illustrating a method for training a federated learning model in accordance with another embodiment of the present application;
FIG. 7 is a schematic diagram of a federated learning model training apparatus according to one embodiment of the present application;
FIG. 8 is a schematic diagram of a Federal learning model training apparatus according to another embodiment of the present application; and
fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary and intended to be used for explaining the present application and should not be construed as limiting the present application.
The method, apparatus, electronic device, and storage medium for training the federal learning model according to an embodiment of the present application are described below with reference to the accompanying drawings.
The method for training the federal learning model provided in the embodiment of the present application may be executed by an electronic device, where the electronic device may be a PC (Personal Computer), a tablet Computer, a server, or the like, and is not limited herein.
In the embodiment of the application, the electronic device can be provided with a processing component, a storage component and a driving component. Optionally, the driver component and the processing component may be integrated, the storage component may store an operating system, an application program, or other program modules, and the processing component implements the method for training the federal learning model provided in this embodiment by executing the application program stored in the storage component.
The method for training the federal learning model provided in the embodiment of the application can be a method for training the federal learning model fusing a bagging (guided aggregation algorithm) -based random forest and GradientBoosting (gradient boosting), wherein each decision tree sub-model in the schemes such as GBDT (machine learning algorithm) can be replaced by a forest composed of a plurality of decision trees, and gradient boosting is performed on each layer of forest. The use of forests instead of a single decision tree has the following advantages:
(1) the robustness is better: on the premise of parallelism, a random forest selects partial data and partial characteristics establish a plurality of decision trees, even if individual decision trees cause poor model effect due to influence of abnormal values, the final output of the forest is a comprehensive result of the plurality of decision trees (in the embodiment of the application, the output of the random forest can be an average value output by each tree model and is not voting of classification results), and therefore the model has better robustness to the abnormal values;
(2) the overfitting problem is alleviated: random forest can be randomly sampled in both characteristic and sample dimension, and Breima (Blima) indicates the upper bound of generalization error of random forest in related paper as
Figure BDA0003298407390000091
Wherein
Figure BDA0003298407390000092
Can be used for fitting weighted correlation coefficient, PE, of residual error item of decision tree in forest*The (tree) can be the average generalization error of the decision trees in the forest, and the formula indicates that a random forest with small generalization error needs to ensure low correlation between the residuals of the decision trees and low generalization error of a single decision tree at the same time. Therefore, the indexes of sample sampling rate, sample characteristics, decision tree depth and decision tree number are considered in theoretical analysis, and abundant and diverse decision trees are constructed through the indexes, so that the similarity among classifiers is reduced, and the overfitting problem is further controlled.
In the embodiment of the application, each layer of random forest needs to determine samples, characteristics, the number of decision trees and the like, and the decision trees are constructed on the determination results. In order to improve modeling efficiency, the method for training the federal learning model provided in the embodiment of the application mainly surrounds two core parameters, namely a decision tree of each layer of forest and a sample sampling rate of each layer of forest, and controls the change of the two core parameters by using an attenuation and increment strategy of cosine annealing. In theoretical analysis and actual modeling experiments, the strategy of cosine annealing is superior to a linear attenuation strategy and is more superior to an exponential attenuation strategy, and the reason for the strategy is that a GBF (gradient Boosting forest) algorithm objective function:
Figure BDA0003298407390000093
wherein
Figure BDA0003298407390000094
Represents the output of the m-th random forest, here the average of a plurality of decision trees, where
Figure BDA0003298407390000095
The predicted result of the random forest of the previous t-1 layer is a constant value y in the t roundiIs a real label. In the present application, the minimum Mean Square Error (MSE) may be used as a loss function, namely:
Figure BDA0003298407390000101
due to the fitting process
Figure BDA0003298407390000102
The loss function value of the algorithm GBF is reduced with increasing modeled tree and the reduction rate is slowed down. Therefore, the strategy of cosine annealing of deceleration attenuation is just fit with the change trend of the algorithm, linear attenuation of uniform-speed attenuation is the first time, and exponential attenuation (the base number is more than 1) of acceleration attenuation is the first time, so that the cosine function can be selected to control the change of forest trees (the number of decision trees) and the sample sampling rate of each layer.
The method for training the federal learning model according to an embodiment of the present application is described in detail below with reference to the accompanying drawings:
FIG. 1 is a flow chart illustrating a method for training a federated learning model according to one embodiment of the present application.
The method for training the federated learning model in the embodiment of the present application may also be implemented by the device for training the federated learning model provided in the embodiment of the present application, and the device may be configured in an electronic device to achieve sample alignment with a data provider server, then obtain the feature numbers of a business server and the data provider server, respectively number the features of the business server and the data provider server according to the feature numbers to generate a feature code set, send the feature number and the public key of the data provider server to the data provider server, then obtain a current sample set and a training parameter set of the federated learning model, perform M times of iterative training on the federated learning model according to the current sample set, the training parameter set, and the feature code set, and obtain a target parameter of the federated learning model obtained by the M times of iterative training, therefore, the joint training between the server of the business party and the server of the data provider is more efficient, and the modeling efficiency is further improved.
As a possible situation, the method for training the federal learning model in the embodiment of the present application may also be executed at a server side, where the server may be a cloud server, and the method for training the federal learning model may be executed at a cloud side.
As shown in fig. 1, the method for training the federal learning model may include:
step 101, aligning samples with a data provider server. There may be a plurality of servers according to the provider.
In the embodiment of the present application, a business party (i.e., a business party server) may perform sample alignment with a data provider server through a preset method. The preset method can be calibrated according to actual conditions. For example, since the user groups of the partners do not completely coincide, the common users of both parties can be confirmed without disclosing the respective data by both the service party and the data provider, and without exposing the users that do not overlap each other, using the encryption-based user sample alignment technique (method).
It should be noted that the sample alignment described in this embodiment may also refer to the alignment of the sample positions between the service server and the data provider server, so as to facilitate accurate sample transmission. In addition, during sample alignment, a communication channel (channel) between the service and the data provider server may be established and encrypted.
102, respectively acquiring the feature quantities of the business side server and the data provider server, respectively numbering the features of the business side server and the data provider server according to the feature quantities to generate a feature code set, and sending the feature number and the public key of the data provider server to the data provider server.
In the embodiment of the application, the data provider server can actively send the local feature quantity of the data provider server to the service provider server. That is, the server at the service side needs to know the number of local features of each data provider server, and the server at the service side can directly obtain the number of features of the server at the service side and the data provider server in its own storage space.
Specifically, after the data provider server and the business server have completed sample alignment (e.g., there are m samples), the business server may first generate a pair of keys, a public key and a private key, locally and synchronize the public key to each business server. The business side server can generate a pair of public and private keys (namely, a public key and a private key) according to a preset key generation algorithm, and can respectively carry out numbering and be marked as { p,P' } wherein, p,Is a public key, and p' is a private key, wherein a preset key generation algorithm can be calibrated according to actual conditions.
Then the service side server can directly obtain the total feature number F from the storage space of the service side server, complete numbering on the F features (the numbering rule can be determined by the service side (service side server) and is not limited to sequential numbering), synchronize the numbering information to the data side (service side server) correspondingly holding each feature, namely, each data side only has the numbering information of the feature owned by the data side, and the full-feature numbering information is only held by the service side.
And 103, acquiring a current sample set and a training parameter set of the federated learning model. The training parameter set may include a feature sampling rate, a training sample upper limit value, a training sample lower limit value, a decision tree number upper limit value, a decision tree number lower limit value, a first parameter change speed, and a second parameter change speed.
In the embodiment of the application, it is assumed that there are m samples after the samples of the Host (data side) side (n ≧ 1) and the Guest (business side) side are aligned, and the Guest side holds the label y and own selfGround data XGuest(feature data), the Host parties each only hold data that can be X respectively0，X1，…，Xn. (each Host side has respective local feature data) it needs to be emphasized that, in the modeling process, the Guest side and the Host side share the above m samples, and the m samples can be put into the current sample set to construct the target training set of each layer of random forest.
Specifically, after sending the feature number and the public key of the data provider server to the data provider server, the business server may obtain a current sample set of the federal learning model and obtain a training parameter set of the federal learning model.
It should be noted that the training parameter set described in the above embodiment may be generated in advance and stored in the storage space of the business side server, so as to be called when needed. The training parameter sets described in the above embodiments may also include a decision tree depth.
And 104, performing M times of iterative training on the federated learning model according to the current sample set, the training parameter set and the feature coding set, wherein M is a positive integer greater than 1.
To illustrate the above embodiment, in an embodiment of the present application, as shown in fig. 2, each iterative training may include:
step 201, using the current iterative training in the M iterative training as the nth iterative training, where N is a positive integer smaller than M.
Specifically, after calling (acquiring) the training parameter set of the federal learning model from the storage space of the business side server, the current iterative training in the M iterative training may be used as the nth iterative training, so that the business side server can be directly used subsequently.
Step 202, generating a sample sampling rate according to M, N, the upper limit value of the training sample, the lower limit value of the training sample and the first parameter change speed.
In the embodiment of the present application, the above-mentioned sample sampling rate can be calculated by the following formula (1):
Figure BDA0003298407390000121
wherein f (X) may be a sample sampling rate, VminMay be a lower limit value of the training sample, VmaxMay be an upper limit value of the training sample, btMay be the current iteration training round (i.e., N), b, aboveTMay be the total number of rounds of iterative training (i.e., M as described above), and k may be the first parameter change rate. π may be the circumferential ratio.
It should be noted that formula (1) described in this embodiment may be generated based on a preset parameter incremental change strategy, where the sample sampling rate f (x) may increase with the increase of the number of rounds of iterative training, that is, the sample sampling rate used in each iterative training increases with the increase of the number of rounds of iterative training. In addition, the formula (1) can be stored in the memory space of the business side server in advance so as to be called when needed.
And step 203, generating a target number according to M, N, the upper limit value of the number of decision trees, the lower limit value of the number of decision trees and the second parameter change speed.
In the embodiment of the present application, the target number of plants can be calculated by the following formula (2):
Figure BDA0003298407390000122
wherein f (Y) may be the target number of plants, WminMay be the lower limit of the number of decision trees, WmaxCan be the upper limit value of the number of the decision trees, dtMay be the current iteration training round (i.e., N), d as described aboveTMay be the total number of rounds of iterative training (i.e., M as described above), and p may be the second parameter change rate. π may be the circumferential ratio.
It should be noted that formula (1) described in this embodiment may be generated based on a preset parameter attenuation change strategy, where the target number f (y) may decrease as the number of rounds of iterative training increases, that is, the number of target blocks used in each iterative training decreases as the number of rounds of iterative training increases. In addition, the formula (2) can also be stored in the memory space of the service side server in advance so as to call when necessary
Specifically, after the current iterative training in the M iterative training is regarded as the nth iterative training, the service side server may first call out the above formula (1) and formula (2) from its own storage space, and may generate (calculate) a sample sampling rate based on the formula (1) and according to the above M, N, the training sample upper limit value, the training sample lower limit value, and the first parameter change speed. Then, based on the formula (2), the target number can be generated (calculated) according to the M, N, the upper limit value of the number of decision trees, the lower limit value of the number of decision trees and the second parameter change speed.
It should be noted that the first parameter change speed and the second parameter change speed in equation (1) and equation (2) described in the above embodiments may respectively control the speed at which the sample sampling rate and the target number change at a given () gradient) total number of iterative training rounds.
For example, taking the above target number as an example, assuming that the target number is calculated by the above formula (2) (i.e., the parameter change strategy is an attenuation strategy), the total round of iterative training (gradient boosting) is 11 rounds, and is set in one layer of random forest (i.e., in one iterative training), the maximum number of decision trees is allowed to be 50 (i.e., the upper limit value of the number of decision trees), the minimum number of decision trees is 15 (i.e., the lower limit value of the number of decision trees), if the number change speed p is 1 (i.e., the second parameter change speed), this means that the random number of decision trees per layer of forest will be finally reduced from 50 to 15 from the 1 st round of training (boosting) to the 11 th round of training under the control of the above formula (2); if the speed p of change of the number of plants is 0.5, this means that under the control of the above formula (2), from the 1 st round of training to the 6 th round of training, the number of decision trees of each layer of random forest will be reduced from 50 to 15, and from the 7 th round of training to the 11 th round of training, the number of decision trees of each layer of random forest will be kept unchanged at 15. The above formula (1) is analogized, and details are not described herein.
It should be further explained that in the present application, only two types of parameters, namely the random forest sample sampling rate and the random forest decision tree number, use a parameter change strategy, and the remaining parameters are fixed values. Meanwhile, in order to reduce algorithm complexity and avoid the situation that a single-layer random forest is not only a large number but also a large number of samples, the random forest sample sampling rate is just opposite to the random forest decision tree number change strategy, namely the sample sampling rate selects an increasing strategy (namely the formula (1)), the number selects an attenuation strategy (namely the formula (2)), otherwise, the sample sampling rate selects the attenuation strategy, and the number selects the increasing strategy.
To fully illustrate the meaning of the references used in a layer of random forest (i.e., in one iterative training), see the following partial parameter definition table (table 1):
Figure BDA0003298407390000131
Figure BDA0003298407390000141
TABLE 1
In step 204, samples with sample sampling rates are selected from the current sample set to generate a target training set.
Step 205, selecting feature codes of the feature sampling rate from the feature code set to generate a target feature code set.
Specifically, after generating the sample sampling rate and the target number, the business server may select samples of the sample sampling rate from the current sample set to generate a target training set, and select feature codes of the feature sampling rate from the feature code set to generate a target feature code set. For example, assuming that the sample sampling rate and the feature sampling rate are both 15%, 15% of samples may be selected from the training set to generate the target training set, and 15% of feature codes may be selected from the feature code set to generate the target feature code set.
Step 206, the number of each sample in the target training set and the target feature number of the data provider server in the target feature code set are sent to the data provider server.
And step 207, generating target parameters of the federated learning model according to the target training set, the target feature coding set and the target number.
Specifically, after the business server generates the target training set and the target feature code set, the serial number of each sample in the target training set and the target feature serial number of the data provider server in the target feature code set may be sent to the data provider server. And then the business side server can generate target parameters of the federal learning model according to the target training set, the target feature coding set and the target number.
Further, the service side server may generate a training parameter set in advance, obtain parameter values (e.g., a sample sampling rate, a feature sampling rate, a number of decision trees, etc.) of the random forest modeling of the local layer according to the parameter change policy (e.g., a parameter incremental change policy and a parameter attenuated change policy), and locally complete random sampling according to the two parameter values of the sample sampling rate and the feature sampling rate based on the aligned sample numbers, thereby determining the sample numbers and the feature numbers participating in the modeling of the current round, and synchronizing related information to the corresponding data provider server. For example, only 10% (sample sampling rate is 10%) of samples participate in the modeling, the Guest party (service party server) needs to synchronize to IDs of 10% of samples participating in the modeling of each Host party (data provider server), and only sqrt (f) number of features participate in the modeling, where after the Guest party randomly samples the features to obtain sqrt (f) selected features, the corresponding feature information needs to be synchronized to each corresponding Host party.
To illustrate the above embodiment clearly, in an embodiment of the present application, as illustrated in fig. 3, generating target parameters of the federal learning model according to the target training set, the target feature coding set and the target number of plants may include:
step 301, calculating gradient information of samples in the target training set, and sending the gradient information to the data provider server.
In one embodiment of the present application, calculating gradient information of samples in the target training set may include generating a first gradient value and a second gradient value of the samples in the target training set, and homomorphically encrypting the first gradient value and the second gradient value to generate the gradient information.
Specifically, the business server may first generate a gradient value g of the samples (i.e., aligned samples) in the target training set according to a preset gradient generation algorithm1And a second order gradient value h1And for a step gradient value g1And a second order gradient value h1Homomorphic encryption to generate gradient information<g1>，<h1>And the gradient information is combined<g1>，<h1>And sending the data to a data provider server. The preset gradient generation algorithm can be calibrated according to actual conditions.
Further, in this embodiment of the present application, the number of samples in the target training set may be multiple, and the server at the service side may generate a first gradient value and a second gradient value (g) for each sample according to a preset gradient generation algorithm1,h1),...,(gn,hn) Then by homomorphic encryption to obtain<g1>,<h1>),...,(<gn>,<hn>) And sending the data to a data provider server, wherein n can be a positive integer.
In the embodiment of the application, the data provider server may receive the feature number and the public key of the data provider server sent by the service provider server, receive the number of each sample in the target training set sent by the service provider server, the target feature number of the data provider server in the target feature coding set, receive the gradient information of the currently trained sample sent by the service provider server, obtain the gradient return information according to the gradient information, and then send the gradient return information to the service provider server.
The obtaining of the gradient return information according to the gradient information may include determining a feature set according to a target feature number and a feature number of a data provider server, splitting a sample space according to a splitting threshold corresponding to each feature in the feature set to obtain a splitting space on an assigned side, then obtaining gradient summation information of the splitting space on the assigned side corresponding to each feature according to the gradient information, numbering the gradient summation information, and generating the gradient return information by using the gradient summation information and the gradient summation information. After the gradient summation information is numbered, generating the number, and mapping relationships among the features corresponding to the number, the splitting threshold and the gradient summation information corresponding to the number may also be included.
Specifically, after receiving the feature number of the data provider server sent by the server of the service provider, the data provider server may know the numbers of all local features of the data provider server, and it should be noted that, after the server of the service provider and the server of the data provider perform sample alignment, both the server of the service provider and the server of the data provider may know the numbers of aligned samples. Then, after the data provider server receives the number of each sample in the target training set sent by the server of the service provider and the target feature number of the server of the data provider in the target feature coding set, the data provider server may determine (acquire) a sample required for a subsequent operation according to the number of the sample, and determine (acquire) a feature required for the subsequent operation according to the target feature number.
Further, after receiving the gradient information of the currently trained sample sent by the server at the service provider, the server at the data provider may determine a feature set according to the target feature number and the feature number of the server at the data provider, that is, select a feature corresponding to the target feature number from features corresponding to the feature number of the server at the data provider according to the target feature number, so as to form the feature set. Then, the data provider server may split the sample space according to the splitting threshold corresponding to each feature in the feature set to obtain a splitting space on the designated side, and obtain gradient summation information of the splitting space on the designated side corresponding to each feature according to the gradient information, that is, perform binning operation, and obtain gradient summation information of the splitting space on the designated side corresponding to each feature according to the gradient information, that is, calculate gradient summation information of a sample in each bin, for example, calculate gradient summation information in a splitting space on the left side (that is, a left space) by the following formulas (1) and (2):
Figure BDA0003298407390000161
Figure BDA0003298407390000162
wherein,<GL>the information may be summed for the first order gradient of the sample,<HL>the information may be summed for the second order gradient of the sample,<gi>may be the first order gradient information of the sample,<hi>may be first order gradient information of the sample, I may be a positive integer less than or equal to n, ILThere may be a split space on the left (i.e., a space of i samples).
The data provider server may then number the gradient summation information and generate gradient return information using the gradient summation information and the numbering of the gradient summation information.
Further, after numbering the gradient summation information, the data provider server may further generate the number, and a mapping relationship between the feature corresponding to the number, the splitting threshold, and the gradient summation information corresponding to the number, and may generate a table. For example, the following mapping in table 2 (i.e., number-feature-split threshold-gradient summation information table):
Figure BDA0003298407390000171
TABLE 2
It should be noted that the gradient return information described in this embodiment may include the number and the gradient summation information.
Finally, the data provider server may send (synchronize) the gradient return information to the server of the business party. Wherein the data provider server may encrypt data sent (synchronized) to the server of the business party.
Step 302, gradient return information provided by the data provider server is received.
And 303, generating a target split point number according to the gradient return information and the target feature coding set, generating a ciphertext based on the private key and the target split point number, and sending the ciphertext to the data provider server.
In an embodiment of the present application, the gradient return information may be multiple, and each gradient return information corresponds to a corresponding number, wherein generating the target split point number according to the gradient return information and the target feature encoding set may include generating a plurality of corresponding information gains according to the multiple gradient return information and the target feature encoding set, respectively, and selecting a maximum information gain from the plurality of information gains, and taking the number corresponding to the maximum information gain as the target split point number.
Specifically, after receiving the gradient return information, the service server may generate a plurality of corresponding information gains according to the gradient return information and the target feature encoding set, and select a maximum information gain from the plurality of information gains, and use a number corresponding to the maximum information gain as a target split point number.
For example, after receiving the gradient return information, the service server may analyze the gradient return information to obtain gradient summation information G for each feature and corresponding binning combinationL、GR、HL、HRWherein G isLInformation, H, which may be a first order gradient sum of samples in a split space on the left (i.e., left space)LInformation, G, may be summed for the second order gradient of samples in the split space on the left (i.e., left space)RInformation, H, may be summed for the first order gradient of samples in the split space on the right (i.e., the right space)RThe information may be summed for the second order gradient of the samples in the split space on the right side (i.e., the right space). Thereby calculating the characteristic information gain of each decision tree node. And after the server at the service side obtains the result of the gradient summation of the data providing side server boxes, the target characteristic coding set is combined, and the plurality of information gains can be calculated through related formulas.
Then, the service server may compare the information gains corresponding to each feature, and select the maximum value, that is, select the maximum information gain from the information gains. It should be noted that the determination may be performed based on a target feature encoding set, and if the feature ID and the threshold ID participating in the above calculation are the feature and the threshold of the data side, the feature number information needs to be sent to the data side, and the sample space is segmented by using the feature returned by the data side; if the business side, the method can be directly split.
Further, the traffic side server finds the maximum information gain (i.e., the first order gradient information gain and the second order gradient information gain) among the above information gains and corresponds to the number q (i.e., the target split point number) in table 2. The business side server may then generate a ciphertext based on the private key and the target split point number, and send the ciphertext to the data provider side server. It should be noted that, the ciphertext may be generated according to the private key and the target split point number based on the related art, which is not described herein again.
In this embodiment, the data provider server may receive a ciphertext generated based on the private key and the target split point number, where the ciphertext is generated according to the gradient return information and the target feature encoding set, and decrypt the ciphertext based on the public key to obtain a decryption operation value, and send the decryption operation value to the service provider server.
Specifically, after receiving the ciphertext sent by the server at the business side, the data provider server may decrypt the ciphertext using the public key to obtain a decryption operation value, and send the decryption operation value to the server at the business side. It should be noted that the public key may be used to decrypt the ciphertext to obtain the decrypted operation value based on the related art, which is not described herein again.
And step 304, receiving the decryption operation value sent by the data provider server, and splitting nodes according to the decryption operation value.
In one embodiment of the present application, as illustrated in fig. 4, performing node splitting according to the decryption operation value may include:
step 401, generating split space information according to the decryption operation value.
And step 402, node splitting is carried out according to the samples in the target training set and the splitting space information.
Specifically, after receiving the decryption operation value sent by the data provider server, the service provider server may generate split space information according to the decryption operation value based on a related formula, where the split space information may be space information required by the service provider server. Then, the service side server may perform a difference set operation to obtain split space information of one side of the optimal split feature according to the sample information in the model training set and the split space information of one side of the optimal split feature (i.e., the split space information described above), thereby completing node splitting (i.e., first node splitting).
And 305, repeating the steps until the model converges to establish a decision tree of the target number, finishing the training of the federal learning model, and acquiring target parameters through the trained federal learning model.
Specifically, the business side server may repeat the above steps 301 to 304 until the model converges to build a decision tree of the target number of plants, completing the training of the federal learning model. It should be noted that, because the random forest can establish a plurality of decision trees in parallel, the modeling of the random forest can also be completed simultaneously according to the above method. And then the business side server can obtain the target parameters through the federal learning model which completes the training.
Therefore, the method for training the federal learning model provided in the embodiment of the present application can generate the target parameters according to the target training set, the target feature coding set and the target number based on the method described in fig. 3.
And 208, generating an optimization label of the current sample based on a gradient lifting algorithm according to the target parameters and the federal learning model, wherein the optimization label is a training label of the current sample of the next round of iterative training.
Specifically, after the iterative training of the current round is finished, the business side server can generate an optimization label of the current sample based on a gradient lifting algorithm according to the target parameters and the federal learning model, use the optimization label as a training label of the current sample of the next iterative training, and then continue to perform the next iterative training.
And 105, obtaining target parameters of the federated learning model obtained by the Mth iterative training.
Specifically, after M iterative trainings, the business side server may obtain target parameters of the federal learning model obtained by the M iterative trainings, that is, obtain target parameters of the federal learning model for completing all iterative trainings.
Therefore, the joint training between the server of the business side and the server of the data provider side is more efficient, the modeling effect is improved, and the complexity is reduced while the model is prevented from being over-fitted by adopting an early-stop strategy.
In the embodiment of the present application, referring to fig. 5, a federal modeling method that integrates two integration strategies, namely a random forest (bagging) and a gradient boosting (boosting) is fused, and a parameter attenuation strategy based on a cosine function and a parameter increment strategy based on the cosine function dynamically optimize parameters of the random forest (taking a sample sampling rate and a number of decision trees as examples), so as to reduce complexity of modeling while ensuring a modeling effect, specifically:
firstly, the model is subjected to gradient promotion on the basis of the random forest, the problem that the random forest model has relatively large deviation is weakened, and time inefficiency is not brought compared with models such as GBDT (guaranteed bit differential transformation) and the like because the random forest can be constructed in parallel. And because the bagging-based random forest modeling effect is superior to that of a single decision tree sub-model, the modeling effect can be improved on the boosting integrated models such as the original GBDT and the like.
Secondly, the application dynamically optimizes the parameters of each layer of forest. Taking the sample sampling rate and the number of decision trees as an example, in the scheme of the application, forest parameters of each layer in the whole model are not consistent, but the number of the decision trees is optimized layer by layer based on a parameter attenuation strategy of a cosine function, and the sample sampling rate is optimized layer by layer based on a parameter increasing strategy of the cosine function. In the conventional parameter search scheme, the entire model needs to be operated on the premise of each possible parameter combination, and the optimal parameter is finally determined. The traditional parameter searching method brings higher time cost, and the application can enable the joint training between the server of the business party and the server of the data provider to be more efficient, thereby improving the modeling efficiency and quality.
Furthermore, the federate learning model training method provided in the embodiment of the present application provides an idea of dynamically optimizing two core parameters, namely, a decision tree of each layer of forest and a sample sampling rate of each layer of forest, so as to reduce complexity, and selects a cosine annealing strategy as an optimal parameter dynamic control strategy after trying three strategies, namely, linearity, exponential and cosine annealing. According to the idea and the method, the algorithm is tried, only the change interval and the change speed of additional parameters need to be set, and a large number of super parameters are not introduced, so that the complexity of the algorithm is reduced, and the modeling effect of the algorithm is guaranteed.
To sum up, according to the method for training the federated learning model in the embodiment of the present application, first, sample alignment is performed with a data provider server, then, feature quantities of a business side server and the data provider server are respectively obtained, features of the business side server and the data provider server are respectively numbered according to the feature quantities to generate a feature code set, the feature number and a public key of the data provider server are sent to the data provider server, then, a current sample set and a training parameter set of the federated learning model are obtained, M times of iterative training are performed on the federated learning model according to the current sample set, the training parameter set and the feature code set, and finally, a target parameter of the federated learning model obtained by the M times of iterative training is obtained. Therefore, the modeling complexity can be reduced while the modeling effect is ensured, so that the joint training between the server of the business party and the server of the data provider is more efficient, and the modeling efficiency is improved.
FIG. 6 is a flow chart illustrating a method for training a federated learning model according to another embodiment of the present application.
The method for training the federal learning model in the embodiment of the application can be further executed by a training device of the federal learning model provided by the embodiment of the application, the device can be configured in electronic equipment to align samples with a server of a business party, receive a feature number of the server of a data provider, a public key, a number of each sample in a target training set, a target feature number of the server of a data provider in a target feature coding set and gradient information of a currently trained sample which are sent by the server of the business party, acquire gradient return information according to the gradient information, send the gradient return information to the server of the business party, receive a ciphertext generated based on a private key and a target split point number and sent by the server of the business party, decrypt the ciphertext based on the public key to obtain a decrypted operation value and send the decrypted operation value to the server of the business party, thereby enabling joint training between the server of the business party and the server of the data provider to be more efficient, and further improves the modeling efficiency.
As a possible situation, the method for training the federal learning model in the embodiment of the present application may also be executed at a server side, where the server may be a cloud server, and the method for training the federal learning model may be executed at a cloud side.
As shown in fig. 6, the method for training the federal learning model may include:
step 601, aligning the sample with the service side server.
In the embodiment of the present application, the data side (i.e., the data provider server) may perform sample alignment with the service side server by a preset method. The preset method can be calibrated according to actual conditions. For example, since the user groups of the partners do not completely coincide, the common users of both parties can be confirmed without disclosing the respective data by both the service party and the data provider, and without exposing the users that do not overlap each other, using the encryption-based user sample alignment technique (method).
It should be noted that the sample alignment described in this embodiment may also refer to the alignment of the sample positions between the service server and the data provider server, so as to facilitate accurate sample transmission. In addition, during sample alignment, a communication channel (channel) between the service and the data provider server may be established and encrypted.
Step 602, receiving the feature number and the public key of the data provider server sent by the service server.
Step 603, receiving the number of each sample in the target training set sent by the service server and the target feature number of the data provider server in the target feature code set.
Step 604, receiving gradient information of the currently trained sample sent by the service side server, and obtaining gradient return information according to the gradient information.
Step 605, sending the gradient return information to the service side server.
And 606, receiving a cipher text generated based on the private key and the target split point number sent by the service side server, wherein the target split point number is generated according to the gradient return information and the target feature coding set.
And step 607, decrypting the ciphertext based on the public key to obtain a decrypted operation value, and sending the decrypted operation value to the service side server.
In one embodiment of the present application, obtaining gradient return information from gradient information includes: determining a feature set according to the target feature number and the feature number of the data provider server; splitting the sample space according to a splitting threshold value corresponding to each feature in the feature set to obtain a splitting space on the designated side; acquiring gradient summation information of the splitting space of the designated side corresponding to each feature according to the gradient information, and numbering the gradient summation information; and generating gradient return information by using the gradient summation information and the number of the gradient summation information.
In an embodiment of the present application, after numbering the gradient summation information, the method further includes: and generating the number, and mapping relation among the feature corresponding to the number, the splitting threshold and the gradient summation information corresponding to the number.
It should be noted that, for details that are not disclosed in the method for training the federal learning model in the embodiment of the present application, please refer to details disclosed in the method for training the federal learning model in the embodiments of fig. 1 to 5 of the present application, and detailed description thereof is omitted here.
To sum up, according to the method for training the federal learning model in the embodiment of the application, firstly, sample alignment is performed with a business side server, then, a feature number and a public key of a data provider server sent by the business side server are received, a number of each sample in a target training set sent by the business side server, a target feature number of the data provider server in a target feature coding set are received, gradient information of a currently trained sample sent by the business side server is received, gradient return information is obtained according to the gradient information, then, the gradient return information is sent to the business side server, a ciphertext generated based on the private key and a target split point number and sent by the business side server is received, finally, the ciphertext is decrypted based on the public key, a decrypted operation value is obtained, and the decrypted operation value is sent to the business side server. Therefore, the modeling complexity can be reduced while the modeling effect is ensured, so that the joint training between the server of the business party and the server of the data provider is more efficient, and the modeling efficiency is improved.
FIG. 7 is a schematic structural diagram of a training apparatus for a federated learning model, according to one embodiment of the present application.
The training device of the federated learning model in the embodiment of the application can be configured in electronic equipment to realize sample alignment with a data provider server, then respectively acquire the feature quantities of a business side server and the data provider server, respectively number the features of the business side server and the data provider server according to the feature quantities to generate a feature code set, send the feature number and a public key of the data provider server to the data provider server, then acquire a current sample set and a training parameter set of the federated learning model, perform M times of iterative training on the federated learning model according to the current sample set, the training parameter set and the feature code set, and acquire a target parameter of the federated learning model obtained by the M times of iterative training, so that the federated training between the business side server and the data provider server is more efficient, and further improves the modeling efficiency.
As shown in fig. 7, the training apparatus 700 of the federal learning model may include: alignment module 710, sending module 720, first acquisition module 730, iterative training module 740, and second acquisition module 750.
The alignment module 710 is configured to perform sample alignment with the data provider server.
The sending module 720 is configured to obtain the feature numbers of the server at the business side and the server at the data provider side, number the features of the server at the business side and the server at the data provider side according to the feature numbers, generate a feature code set, and send the feature number and the public key of the server at the data provider side to the server at the data provider side.
The first obtaining module 730 is configured to obtain a current sample set and a training parameter set of the federated learning model.
The iterative training module 740 is configured to perform M times of iterative training on the joint learning model according to the current sample set, the training parameter set, and the feature coding set, where M is a positive integer greater than 1.
The second obtaining module 750 is configured to obtain target parameters of the federal learning model obtained by the mth iterative training.
In one embodiment of the present application, the training parameter set includes a feature sampling rate, a training sample upper limit value, a training sample lower limit value, a decision tree number upper limit value, a decision tree number lower limit value, a first parameter change speed, and a second parameter change speed.
In one embodiment of the present application, iterative training module 740 may comprise: the device comprises a setting submodule, a first generating submodule, a second generating submodule, a third generating submodule, a fourth generating submodule, a sending submodule, a fifth generating submodule and a sixth generating submodule.
And the setting submodule is used for taking the current iterative training in M iterative training times as the Nth iterative training time, wherein N is a positive integer smaller than M.
And the first generation submodule is used for generating a sample sampling rate according to M, N, the upper limit value of the training sample, the lower limit value of the training sample and the first parameter change speed.
And the second generation submodule is used for generating the target number according to M, N, the upper limit value of the number of the decision trees, the lower limit value of the number of the decision trees and the second parameter change speed.
And the third generation sub-module is used for selecting samples of the sample sampling rate from the current sample set to generate a target training set.
And the fourth generation submodule is used for selecting the feature codes of the feature sampling rate from the feature code set to generate a target feature code set.
And the sending submodule is used for sending the number of each sample in the target training set and the target feature number of the data provider server in the target feature coding set to the data provider server.
And the fifth generation submodule is used for generating target parameters of the federal learning model according to the target training set, the target feature coding set and the target number.
And the sixth generation submodule is used for generating an optimization label of the current sample based on a gradient lifting algorithm and according to the target parameter and the federal learning model, wherein the optimization label is a training label of the current sample of the next round of iterative training.
In an embodiment of the present application, the fifth generation submodule may include: the device comprises a calculation unit, a receiving unit, a generation unit, a node splitting unit and an acquisition unit.
The calculating unit is used for calculating the gradient information of the samples in the target training set and sending the gradient information to the data providing server.
And the receiving unit is used for receiving the gradient return information provided by the data provider server.
And the generating unit is used for generating a target split point number according to the gradient return information and the target feature coding set, generating a ciphertext based on the private key and the target split point number, and sending the ciphertext to the data provider server.
And the node splitting unit is used for receiving the decryption operation value sent by the data provider server and splitting the node according to the decryption operation value.
And the obtaining unit is used for repeating the steps until the model converges to establish a decision tree of the target number, finish the training of the federal learning model and obtain the target parameters through the federal learning model which finishes the training.
In an embodiment of the present application, the computing unit is specifically configured to: generating a first-order gradient value and a second-order gradient value of the samples in the target training set; homomorphic encryption is performed on the first gradient value and the second gradient value to generate gradient information.
In an embodiment of the present application, the gradient return information is multiple, and each gradient return information corresponds to a corresponding number, wherein the generating unit is specifically configured to: respectively generating a plurality of corresponding information gains according to the gradient return information and the target feature coding set; the maximum information gain is selected from the plurality of information gains, and the number corresponding to the maximum information gain is used as the target split point number.
In an embodiment of the present application, the node splitting unit is specifically configured to: generating split space information according to the decryption operation value; and splitting the nodes according to the samples in the target training set and the splitting space information.
It should be noted that, for details that are not disclosed in the training apparatus of the federal learning model in the embodiment of the present application, please refer to details disclosed in the training method of the federal learning model in the embodiments of fig. 1 to 5 of the present application, and detailed description thereof is omitted here.
In summary, the training apparatus of the federal learning model in the embodiment of the present application performs sample alignment with the data provider server through the alignment module, then the characteristic quantities of the server of the business party and the server of the data provider are respectively obtained through a sending module, and numbering the characteristics of the server of the service provider and the server of the data provider according to the characteristic quantity respectively to generate a characteristic code set, and the characteristic number and the public key of the data provider server are sent to the data provider server, then the current sample set and the training parameter set of the federated learning model are obtained through the first obtaining module, and the iterative training module is used for obtaining the code set according to the current sample set, the training parameter set and the characteristic code set, and performing iterative training on the federated learning model for M times, and finally obtaining target parameters of the federated learning model obtained by the iterative training for the M time through a second obtaining module. Therefore, the modeling complexity can be reduced while the modeling effect is ensured, so that the joint training between the server of the business party and the server of the data provider is more efficient, and the modeling efficiency is improved.
Fig. 8 is a schematic structural diagram of a training apparatus of the federal learning model according to another embodiment of the present application.
The Federal learning model training device of the embodiment of the application can be configured in electronic equipment to realize sample alignment with a business side server, and receives the feature number of the data provider server, the public key, the number of each sample in the target training set, the target feature number of the data provider server in the target feature coding set and the gradient information of the currently trained sample, which are sent by the service provider server, and acquires the gradient return information according to the gradient information, then sending gradient return information to the server of the service party, receiving a cipher text which is sent by the server of the service party and generated based on the private key and the number of the target split point, and decrypting the ciphertext based on the public key to obtain a decrypted operation value, and sending the decrypted operation value to the service side server, therefore, the joint training between the server of the business party and the server of the data provider is more efficient, and the modeling efficiency is further improved.
As shown in fig. 8, the training apparatus 800 of the federal learning model may include: an alignment module 810, a first receiving module 820, a second receiving module 830, a third receiving module 840, a first transmitting module 850, a fourth receiving module 860, and a second transmitting module 870.
The alignment module 810 is configured to perform sample alignment with the server at the service side.
The first receiving module 820 is configured to receive the feature number and the public key of the data provider server sent by the server at the service provider.
The second receiving module 830 is configured to receive a number of each sample in the target training set sent by the server at the service provider, and a target feature number of the server at the data provider in the target feature coding set.
The third receiving module 840 is configured to receive gradient information of a currently trained sample sent by the server at the service side, and obtain gradient return information according to the gradient information.
The first sending module 850 is configured to send the gradient return information to the server of the service party.
The fourth receiving module 860 is configured to receive a ciphertext generated based on the private key and a target split point number, where the ciphertext is sent by the server at the service side, and the target split point number is generated according to the gradient return information and the target feature encoding set.
The second sending module 870 is configured to decrypt the ciphertext based on the public key to obtain a decrypted operation value, and send the decrypted operation value to the service side server.
In an embodiment of the present application, the third receiving module 840 is specifically configured to: determining a feature set according to the target feature number and the feature number of the data provider server; splitting the sample space according to a splitting threshold value corresponding to each feature in the feature set to obtain a splitting space on the designated side; acquiring gradient summation information of the splitting space of the designated side corresponding to each feature according to the gradient information, and numbering the gradient summation information; and generating gradient return information by using the gradient summation information and the number of the gradient summation information.
In an embodiment of the present application, the third receiving module 840 is further configured to: and generating the number, and mapping relation among the feature corresponding to the number, the splitting threshold and the gradient summation information corresponding to the number.
It should be noted that, for details that are not disclosed in the training apparatus of the federal learning model in the embodiment of the present application, please refer to details disclosed in the training method of the federal learning model in the embodiments of fig. 1 to 5 of the present application, and detailed description thereof is omitted here.
To sum up, the training apparatus of the federal learning model in the embodiment of the present application performs sample alignment with the server at the service side through the alignment module, then receives the feature number and the public key of the server at the data provider side sent by the server at the service side through the first receiving module, receives the number of each sample in the target training set sent by the server at the service side and the target feature number of the server at the target feature code set through the second receiving module, receives the gradient information of the currently trained sample sent by the server at the service side through the third receiving module, obtains the gradient return information according to the gradient information, then sends the gradient return information to the server at the service side through the first sending module, and receives the ciphertext generated based on the private key and the target split point number sent by the server at the service side through the fourth receiving module, and finally, the ciphertext is decrypted through the second sending module based on the public key to obtain a decrypted operation value, and the decrypted operation value is sent to the service side server. Therefore, the modeling complexity can be reduced while the modeling effect is ensured, so that the joint training between the server of the business party and the server of the data provider is more efficient, and the modeling efficiency is improved.
In order to implement the foregoing embodiments, as shown in fig. 9, the present application further proposes an electronic device 900, which includes a memory 910, a processor 920 and a computer program stored in the memory 910 and being executable on the processor 920, where the processor 920 executes the program to implement the method for training the federal learning model proposed in the foregoing embodiments of the present application.
According to the electronic equipment, the processor executes the computer program stored on the memory, the modeling effect can be guaranteed, and meanwhile the modeling complexity can be reduced, so that the joint training between the server of the business party and the server of the data provider is more efficient, and the modeling efficiency is improved.
In order to implement the foregoing embodiments, the present application further proposes a non-transitory computer-readable storage medium, on which a computer program is stored, the program being executed by a processor to implement the method for training the federal learning model proposed in the foregoing embodiments of the present application.
The computer-readable storage medium of the embodiment of the application can reduce the complexity of modeling while ensuring the modeling effect by storing the computer program and being executed by the processor, so that the joint training between the server of the business party and the server of the data provider is more efficient, and the modeling efficiency is further improved.
In the description of the present specification, the terms "first", "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implying any number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present application, "plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
In the description herein, reference to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Although embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application, and that variations, modifications, substitutions and alterations may be made to the above embodiments by those of ordinary skill in the art within the scope of the present application.

Claims (22)

1. A method for training a federated learning model is characterized by comprising the following steps:
sample alignment with a data provider server;
respectively acquiring the feature quantity of a business side server and the feature quantity of a data provider server, respectively numbering the features of the business side server and the data provider server according to the feature quantity to generate a feature code set, and sending the feature number and a public key of the data provider server to the data provider server;
acquiring a current sample set and a training parameter set of a federated learning model;
performing iterative training on the federated learning model for M times according to the current sample set, the training parameter set and the feature coding set, wherein M is a positive integer greater than 1; and
and obtaining target parameters of the federated learning model obtained by the Mth iterative training.
2. A method of training a federal learning model as defined in claim 1, wherein the set of training parameters includes a feature sampling rate, a training sample upper limit value, a training sample lower limit value, a decision tree number upper limit value, a decision tree number lower limit value, a first parameter change rate, and a second parameter change rate.
3. A method for training a federal learning model as claimed in claim 2, wherein each iteration of training comprises:
taking the current iterative training in the M iterative training as the Nth iterative training, wherein N is a positive integer smaller than M;
generating a sample sampling rate according to the M, the N, the training sample upper limit value, the training sample lower limit value and the first parameter change speed;
generating a target number according to the M, the N, the upper limit value of the number of decision trees, the lower limit value of the number of decision trees and the second parameter change speed;
selecting samples of the sample sampling rate from the current sample set to generate a target training set;
selecting feature codes of the feature sampling rate from the feature code set to generate a target feature code set;
sending the number of each sample in the target training set and the target feature number of the data provider server in the target feature coding set to the data provider server;
generating target parameters of the federated learning model according to the target training set, the target feature coding set and the target number;
and generating an optimization label of the current sample based on a gradient lifting algorithm and according to the target parameter and the federal learning model, wherein the optimization label is a training label of the current sample of the next round of iterative training.
4. The method for training a federal learning model as claimed in claim 3, wherein the generating target parameters of the federal learning model according to the target training set, the target feature coding set and the target number of plants comprises:
calculating gradient information of the samples in the target training set, and sending the gradient information to the data provider server;
receiving gradient return information provided by the data provider server;
generating a target split point number according to the gradient return information and the target feature coding set, generating a ciphertext based on a private key and the target split point number, and sending the ciphertext to the data provider server;
receiving a decryption operation value sent by the data provider server, and splitting nodes according to the decryption operation value;
repeating the steps until the model converges to establish a decision tree of the target number, finishing the training of the federated learning model, and obtaining the target parameters through the federated learning model which finishes the training.
5. The method for training a federal learning model as claimed in claim 4, wherein said calculating gradient information for samples in the target training set comprises:
generating a first-order gradient value and a second-order gradient value of the samples in the target training set;
homomorphically encrypting the first-order gradient value and the second-order gradient value to generate the gradient information.
6. The method for training a federal learning model as claimed in claim 4, wherein the gradient return information includes a plurality of gradient return information, and each gradient return information corresponds to a corresponding number, wherein the generating a target split point number according to the gradient return information and the target feature encoding set includes:
respectively generating a plurality of corresponding information gains according to the gradient return information and the target feature coding set;
and selecting the maximum information gain from the plurality of information gains, and taking the number corresponding to the maximum information gain as the target split point number.
7. The method for training a federated learning model as recited in claim 4, wherein the performing node splitting based on the decrypted computation value comprises:
generating split space information according to the decryption operation value;
and splitting nodes according to the samples in the target training set and the splitting space information.
8. A method for training a federated learning model is characterized by comprising the following steps:
performing sample alignment with a service side server;
receiving the characteristic number and the public key of the data provider server sent by the service party server;
receiving the number of each sample in a target training set sent by the service side server and the target feature number of the data provider server in a target feature coding set;
receiving gradient information of a currently trained sample sent by the service side server, and acquiring gradient return information according to the gradient information;
sending the gradient return information to the service side server;
receiving a cipher text which is sent by the service side server and generated based on a private key and a target split point number, wherein the target split point number is generated according to the gradient return information and the target feature coding set; and
and decrypting the ciphertext based on the public key to obtain a decrypted operation value, and sending the decrypted operation value to the service side server.
9. The method for training a federal learning model as claimed in claim 8, wherein said obtaining gradient return information according to the gradient information comprises:
determining a feature set according to the target feature number and the feature number of the data provider server;
splitting a sample space according to a splitting threshold value corresponding to each feature in the feature set to obtain a splitting space on a designated side;
acquiring gradient summation information of the splitting space of the designated side corresponding to each feature according to the gradient information, and numbering the gradient summation information;
and generating the gradient return information by using the gradient summation information and the serial number of the gradient summation information.
10. The method for training a federal learning model as claimed in claim 9, wherein said numbering said gradient sum information further comprises:
and generating the number and a mapping relation among the feature corresponding to the number, the splitting threshold and the gradient summation information corresponding to the number.
11. The utility model provides a trainer of bang's learning model which characterized in that includes:
the alignment module is used for aligning samples with the data provider server;
the sending module is used for respectively acquiring the feature quantities of the business side server and the data provider side server, respectively numbering the features of the business side server and the data provider side server according to the feature quantities to generate a feature coding set, and sending the feature number and the public key of the data provider side server to the data provider side server;
the first acquisition module is used for acquiring a current sample set and a training parameter set of the federated learning model;
the iterative training module is used for carrying out M times of iterative training on the federated learning model according to the current sample set, the training parameter set and the feature coding set, wherein M is a positive integer greater than 1; and
and the second acquisition module is used for acquiring the target parameters of the federated learning model obtained by the Mth iterative training.
12. The apparatus for training a federal learning model as defined in claim 11, wherein the set of training parameters includes a feature sampling rate, a training sample upper limit value, a training sample lower limit value, a decision tree number upper limit value, a decision tree number lower limit value, a first parameter change rate, and a second parameter change rate.
13. The apparatus for training a federal learning model as claimed in claim 12, wherein said iterative training module comprises:
a setting submodule, configured to use a current iterative training in the M iterative training times as an nth iterative training time, where N is a positive integer smaller than M;
the first generation submodule is used for generating a sample sampling rate according to the M, the N, the upper limit value of the training sample, the lower limit value of the training sample and the first parameter change speed;
a second generation submodule, configured to generate a target number according to the M, the N, the upper limit of the number of decision trees, the lower limit of the number of decision trees, and the second parameter change speed;
the third generation submodule is used for selecting samples of the sample sampling rate from the current sample set so as to generate a target training set;
the fourth generation submodule is used for selecting the feature codes of the feature sampling rate from the feature code set to generate a target feature code set;
a sending submodule, configured to send the number of each sample in the target training set and the target feature number of the data provider server in the target feature coding set to the data provider server;
a fifth generation submodule, configured to generate a target parameter of the federated learning model according to the target training set, the target feature coding set, and the target number;
and a sixth generation submodule, configured to generate an optimization label of the current sample according to the target parameter and the federal learning model based on a gradient boost algorithm, where the optimization label is a training label of the current sample for a next round of iterative training.
14. The apparatus for training a federal learning model as claimed in claim 13, wherein said fifth generation submodule comprises:
the calculation unit is used for calculating the gradient information of the samples in the target training set and sending the gradient information to the data provider server;
the receiving unit is used for receiving the gradient return information provided by the data provider server;
the generating unit is used for generating a target split point number according to the gradient return information and the target feature coding set, generating a ciphertext based on a private key and the target split point number, and sending the ciphertext to the data provider server;
the node splitting unit is used for receiving the decryption operation value sent by the data provider server and splitting nodes according to the decryption operation value;
and the obtaining unit is used for repeating the steps until the model converges to establish a decision tree of the target number, finish the training of the federated learning model and obtain the target parameters through the federated learning model which finishes the training.
15. The apparatus for training a federal learning model as in claim 14, wherein the computing unit is specifically configured to:
generating a first-order gradient value and a second-order gradient value of the samples in the target training set;
homomorphically encrypting the first-order gradient value and the second-order gradient value to generate the gradient information.
16. The apparatus for training a federal learning model as claimed in claim 14, wherein the gradient return information includes a plurality of gradient return information, and each gradient return information corresponds to a corresponding number, wherein the generating unit is specifically configured to:
respectively generating a plurality of corresponding information gains according to the gradient return information and the target feature coding set;
and selecting the maximum information gain from the plurality of information gains, and taking the number corresponding to the maximum information gain as the target split point number.
17. The apparatus for training a federal learning model as in claim 14, wherein the node split unit is specifically configured to:
generating split space information according to the decryption operation value;
and splitting nodes according to the samples in the target training set and the splitting space information.
18. The utility model provides a trainer of bang's learning model which characterized in that includes:
the alignment module is used for aligning samples with the service side server;
the first receiving module is used for receiving the characteristic number and the public key of the data provider server sent by the service party server;
the second receiving module is used for receiving the serial number of each sample in the target training set sent by the server of the service party and the target characteristic serial number of the server of the data provider in the target characteristic coding set;
the third receiving module is used for receiving the gradient information of the currently trained sample sent by the service side server and acquiring gradient return information according to the gradient information;
the first sending module is used for sending the gradient return information to the service side server;
a fourth receiving module, configured to receive a ciphertext generated based on a private key and a target split point number, where the target split point number is generated according to the gradient return information and the target feature encoding set, and the ciphertext is sent by the server of the service party; and
and the second sending module is used for decrypting the ciphertext based on the public key to obtain a decrypted operation value and sending the decrypted operation value to the service side server.
19. The apparatus for training a federal learning model as claimed in claim 18, wherein said third receiving module is specifically configured to:
determining a feature set according to the target feature number and the feature number of the data provider server;
splitting a sample space according to a splitting threshold value corresponding to each feature in the feature set to obtain a splitting space on a designated side;
acquiring gradient summation information of the splitting space of the designated side corresponding to each feature according to the gradient information, and numbering the gradient summation information;
and generating the gradient return information by using the gradient summation information and the serial number of the gradient summation information.
20. The apparatus for training a federal learning model as claimed in claim 19, wherein said third receiving module is further configured to:
and generating the number and a mapping relation among the feature corresponding to the number, the splitting threshold and the gradient summation information corresponding to the number.
21. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing a method for training a federated learning model as recited in any of claims 1-7 or claims 8-10 when executing the program.
22. A computer-readable storage medium on which a computer program is stored, which program, when executed by a processor, implements a method of training a federal learning model as claimed in any of claims 1-7 or claims 8-10.
CN202111183940.5A 2021-10-11 2021-10-11 Federal learning model training method and device, electronic equipment and storage medium Pending CN113947211A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111183940.5A CN113947211A (en) 2021-10-11 2021-10-11 Federal learning model training method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111183940.5A CN113947211A (en) 2021-10-11 2021-10-11 Federal learning model training method and device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN113947211A true CN113947211A (en) 2022-01-18

Family

ID=79329639

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111183940.5A Pending CN113947211A (en) 2021-10-11 2021-10-11 Federal learning model training method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN113947211A (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114118312A (en) * 2022-01-29 2022-03-01 华控清交信息科技(北京)有限公司 Vertical training method, device, electronic equipment and system for GBDT model
CN114118641A (en) * 2022-01-29 2022-03-01 华控清交信息科技(北京)有限公司 Wind power plant power prediction method, GBDT model longitudinal training method and device
CN114448597A (en) * 2022-01-25 2022-05-06 京东科技控股股份有限公司 Data processing method and device, computer equipment and storage medium
CN116822660A (en) * 2023-03-21 2023-09-29 北京火山引擎科技有限公司 Longitudinal federal learning method, device, electronic equipment and readable storage medium
CN117675411A (en) * 2024-01-31 2024-03-08 智慧眼科技股份有限公司 Global model acquisition method and system based on longitudinal XGBoost algorithm
WO2024060906A1 (en) * 2022-09-20 2024-03-28 腾讯科技（深圳）有限公司 Data processing method and apparatus for federated learning system, computer, and readable storage medium
CN117972793A (en) * 2024-03-28 2024-05-03 中电科网络安全科技股份有限公司 Longitudinal federal tree model training method, device, equipment and storage medium

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109165683A (en) * 2018-08-10 2019-01-08 深圳前海微众银行股份有限公司 Sample predictions method, apparatus and storage medium based on federation's training
CN109657055A (en) * 2018-11-09 2019-04-19 中山大学 Title party article detection method and federal learning strategy based on level hybrid network
CN110633806A (en) * 2019-10-21 2019-12-31 深圳前海微众银行股份有限公司 Longitudinal federated learning system optimization method, device, equipment and readable storage medium
CN111598186A (en) * 2020-06-05 2020-08-28 腾讯科技（深圳）有限公司 Decision model training method, prediction method and device based on longitudinal federal learning
CN111614679A (en) * 2020-05-22 2020-09-01 深圳前海微众银行股份有限公司 Federal learning qualification recovery method, device and readable storage medium
CN113722987A (en) * 2021-08-16 2021-11-30 京东科技控股股份有限公司 Federal learning model training method and device, electronic equipment and storage medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109165683A (en) * 2018-08-10 2019-01-08 深圳前海微众银行股份有限公司 Sample predictions method, apparatus and storage medium based on federation's training
CN109657055A (en) * 2018-11-09 2019-04-19 中山大学 Title party article detection method and federal learning strategy based on level hybrid network
CN110633806A (en) * 2019-10-21 2019-12-31 深圳前海微众银行股份有限公司 Longitudinal federated learning system optimization method, device, equipment and readable storage medium
CN111614679A (en) * 2020-05-22 2020-09-01 深圳前海微众银行股份有限公司 Federal learning qualification recovery method, device and readable storage medium
CN111598186A (en) * 2020-06-05 2020-08-28 腾讯科技（深圳）有限公司 Decision model training method, prediction method and device based on longitudinal federal learning
CN113722987A (en) * 2021-08-16 2021-11-30 京东科技控股股份有限公司 Federal learning model training method and device, electronic equipment and storage medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
董业;侯炜;陈小军;曾帅;: "基于秘密分享和梯度选择的高效安全联邦学习", 计算机研究与发展, no. 10, 9 October 2020 (2020-10-09), pages 235 - 244 *

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114448597A (en) * 2022-01-25 2022-05-06 京东科技控股股份有限公司 Data processing method and device, computer equipment and storage medium
CN114118312A (en) * 2022-01-29 2022-03-01 华控清交信息科技(北京)有限公司 Vertical training method, device, electronic equipment and system for GBDT model
CN114118641A (en) * 2022-01-29 2022-03-01 华控清交信息科技(北京)有限公司 Wind power plant power prediction method, GBDT model longitudinal training method and device
CN114118641B (en) * 2022-01-29 2022-04-19 华控清交信息科技(北京)有限公司 Wind power plant power prediction method, GBDT model longitudinal training method and device
CN114118312B (en) * 2022-01-29 2022-05-13 华控清交信息科技(北京)有限公司 Vertical training method, device, electronic equipment and system for GBDT model
WO2024060906A1 (en) * 2022-09-20 2024-03-28 腾讯科技（深圳）有限公司 Data processing method and apparatus for federated learning system, computer, and readable storage medium
CN116822660A (en) * 2023-03-21 2023-09-29 北京火山引擎科技有限公司 Longitudinal federal learning method, device, electronic equipment and readable storage medium
CN117675411A (en) * 2024-01-31 2024-03-08 智慧眼科技股份有限公司 Global model acquisition method and system based on longitudinal XGBoost algorithm
CN117675411B (en) * 2024-01-31 2024-04-26 智慧眼科技股份有限公司 Global model acquisition method and system based on longitudinal XGBoost algorithm
CN117972793A (en) * 2024-03-28 2024-05-03 中电科网络安全科技股份有限公司 Longitudinal federal tree model training method, device, equipment and storage medium

Similar Documents

Publication Publication Date Title
CN113947211A (en) Federal learning model training method and device, electronic equipment and storage medium
CN113722987B (en) Training method and device of federal learning model, electronic equipment and storage medium
CN108632032B (en) Safe multi-keyword sequencing retrieval system without key escrow
CN114401079B (en) Multi-party united information value calculation method, related equipment and storage medium
Doganay et al. Distributed privacy preserving k-means clustering with additive secret sharing
CN107196926B (en) Cloud outsourcing privacy set comparison method and device
CN110084377A (en) Method and apparatus for constructing decision tree
US20070005594A1 (en) Secure keyword search system and method
Li et al. Robust batch steganography in social networks with non-uniform payload and data decomposition
CN112862001A (en) Decentralized data modeling method under privacy protection
WO2007043490A1 (en) Method for securely classifying private data
Kiayias et al. Traitor deterring schemes: Using bitcoin as collateral for digital content
CN112862057B (en) Modeling method, modeling device, electronic equipment and readable medium
CN115841133A (en) Method, device and equipment for federated learning and storage medium
WO2022076826A1 (en) Privacy preserving machine learning via gradient boosting
Kumar Technique for security of multimedia using neural network
CN115834067A (en) Ciphertext data sharing method in edge cloud collaborative scene
CN114172655B (en) Secure multiparty computing data system, method, equipment and data processing terminal
CN111859440B (en) Sample classification method of distributed privacy protection logistic regression model based on mixed protocol
CN110659453B (en) Block chain digital copyright protection method and system based on invention principle
CN108632257B (en) Method and system for acquiring encrypted health record supporting hierarchical search
CN114866236B (en) Data sharing method of Internet of things in cloud based on alliance chain
Fang et al. Flfe: a communication-efficient and privacy-preserving federated feature engineering framework
CN116029392A (en) Joint training method and system based on federal learning
CN114723068A (en) Federal model training method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination