CN113762328A - Model training method, device, equipment and storage medium based on federal learning - Google Patents
Model training method, device, equipment and storage medium based on federal learning Download PDFInfo
- Publication number
- CN113762328A CN113762328A CN202110666784.1A CN202110666784A CN113762328A CN 113762328 A CN113762328 A CN 113762328A CN 202110666784 A CN202110666784 A CN 202110666784A CN 113762328 A CN113762328 A CN 113762328A
- Authority
- CN
- China
- Prior art keywords
- gradient
- electronic device
- feature
- sample space
- random number
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/243—Classification techniques relating to the number of classes
- G06F18/24323—Tree-organised classifiers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Software Systems (AREA)
- Medical Informatics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application discloses a method and a device for model training based on federal learning, and the specific implementation scheme is as follows: when the tree threshold value of the gradient lifting tree is not reached and the depth threshold value of the gradient lifting tree is not reached, sending gradient data to each second electronic device in the federal learning system; receiving random number codes and first fusion gradients of splitting threshold values of each characteristic and corresponding characteristics in a second characteristic information set sent by each second electronic device based on gradient data; determining a target value of the information gain based on each of the first fusion gradients and the first feature information set; in response to the fact that the target value of the information gain is the maximum value of the local information gain, dividing the sample space based on the corresponding characteristic of the target value of the information gain and the splitting threshold value to generate a sample space division result and a target fusion gradient; and updating the model parameters of the first electronic equipment according to the target fusion gradient. The scheme realizes an interpretable model training method based on federal learning.
Description
Technical Field
The embodiment of the application relates to the technical field of computers, in particular to the technical field of machine learning, and particularly relates to a method and a device for model training based on federal learning.
Background
With the vigorous development of big data and artificial intelligence technology, the demand of the internet finance field for big data and artificial intelligence technology is increasing day by day. Federal learning is a distributed machine learning paradigm based on privacy and safety as a modeling scheme for solving the data islanding problem, and gradually receives attention in the industry. Federal learning is generally performed by one party (called a Guest party) providing tag information and part of feature information, and the other party (called a Host party) providing feature information. In the machine learning algorithm, the gradient lifting tree has interpretability of the decision tree model and improves classification performance and effect. Under the requirement of multi-party safety modeling, based on the federal learning technology and the gradient tree-lifting algorithm, a certain bank provides a gradient tree-lifting method based on semi-homomorphic encryption, namely a SecureBoost algorithm. However, in order to prevent sample information from being leaked, modeling features between parties cannot be disclosed, so that model interpretability is reduced, and in the prior art, a data party may destroy a model by providing malicious data, so that great potential safety hazards exist.
Disclosure of Invention
The application provides a model training method, a device, equipment and a storage medium based on federal learning.
According to a first aspect of the application, a model training method based on federal learning is provided, and is applied to first electronic equipment in a federal learning system, wherein each user identifier and a first feature information set of each user are stored in the first electronic equipment, and a model is constructed based on a gradient lifting tree algorithm, and the method includes the following steps: when the tree threshold value of the gradient lifting tree is not reached and the depth threshold value of the gradient lifting tree is not reached, sending gradient data to each second electronic device in the federated learning system, wherein the gradient data are generated based on a prediction result of a built tree of the gradient lifting tree, and a second characteristic information set of a user same as that of the first electronic device is stored in the second electronic device; receiving random number codes representing the splitting threshold value of each feature and the corresponding feature in the second feature information set and first fusion gradients corresponding to the random number codes, which are sent by each second electronic device based on the gradient data; determining a target value of the information gain based on each first fusion gradient and each first characteristic information set, wherein the target value of the information gain is obtained by selecting a maximum value of the local information gain and a maximum value of each second electronic device information gain, the maximum value of the local information gain is used for representing a maximum value of each characteristic in each information gain after each splitting threshold value in each first characteristic information set in the current node sample space, and the maximum value of the second electronic device information gain is obtained by decrypting the corresponding first fusion gradient; in response to the fact that the target value of the information gain is the maximum value of the local information gain, dividing the sample space based on the characteristics corresponding to the target value of the information gain and the splitting threshold value, and generating a sample space division result and a target fusion gradient corresponding to the sample space division result; and updating the model parameters of the first electronic equipment according to the target fusion gradient.
In some embodiments, after determining the target value of the information gain based on the respective first fused gradients and the first feature information set, further comprises: responding to the target value of the information gain as the maximum value of the information gain of the second electronic equipment, and sending a random number code corresponding to the target value of the information gain to the second electronic equipment; and receiving a sample space division result sent by the second electronic equipment and a target fusion gradient corresponding to the sample space division result.
In some embodiments, the gradient data is used to characterize the first and second order gradients of the respective samples; the gradient data is generated by encrypting the first and second order gradients using encryption techniques, including homomorphic encryption techniques.
In some embodiments, the random number encoding is an encrypted random number encoding characterizing a split threshold for each feature and corresponding feature in the second set of feature information.
In some embodiments, the random number codes are generated by scrambling the feature ordering based on the split threshold of each feature identifier and corresponding feature in the second feature information set in the current node sample space, and the random number codes are different from each other.
In some embodiments, the method further comprises: sending the sample space division result to each second electronic device; and/or updating the prediction result of the model based on the sample space division result when the depth threshold of the gradient lifting tree is reached.
In some embodiments, the method further comprises: when the tree threshold of the gradient lifting tree is reached, generating a final model after training is completed, and sending the final model to each second electronic device; and receiving the contribution degrees which are sent by the second electronic equipment based on the final model and correspond to the features in the second feature information set.
According to a second aspect of the present application, there is provided a model training system based on federal learning, the system comprising: a first electronic device, wherein the first electronic device is configured to perform any one of the above methods of model training based on federated learning.
In some embodiments, the system further comprises: at least one second electronic device; the second electronic equipment is used for receiving the gradient data sent by the first electronic equipment in the federal learning system; generating random number codes corresponding to the splitting threshold values of each feature and the corresponding feature in the second feature information set and first fusion gradients corresponding to the random number codes based on the gradient data, and sending the random number codes and the first fusion gradients to the first electronic equipment; in response to receiving the random number code sent by the first electronic device, dividing the sample space based on the corresponding features of the random number code sent by the first electronic device and the splitting threshold value, and generating a sample space division result and a target fusion gradient corresponding to the sample space division result.
In some embodiments, the second electronic device is further configured to send the sample space division result and a target fusion gradient corresponding to the sample space division result to the first electronic device.
In some embodiments, the second electronic device is further configured to, in response to receiving the final model sent by the first electronic device, send contribution degrees corresponding to respective features in the second feature information set to the first electronic device, where the contribution degrees are obtained based on the final model.
According to a third aspect of the present application, there is provided a model training apparatus applied to a first electronic device in a federal learning system, where a first feature information set of each user and each user is stored in the first electronic device, and a model is constructed based on a gradient lifting tree algorithm, the apparatus including: the first sending unit is configured to send gradient data to each second electronic device in the federated learning system when a tree threshold of a gradient lifting tree is not reached and a depth threshold of the gradient lifting tree is not reached, wherein the gradient data are generated based on a prediction result of a built tree of the gradient lifting tree, and a second feature information set of the same user as that of the first electronic device is stored in the second electronic device; a first receiving unit configured to receive, based on the gradient data, random number codes representing splitting thresholds of each feature and a corresponding feature in the second feature information set and first fusion gradients corresponding to the respective random number codes, which are transmitted by the respective second electronic devices; a determining unit configured to determine a target value of the information gain based on each first fusion gradient and each first feature information set, wherein the target value of the information gain is obtained by selecting a maximum value of the local information gain and a maximum value of each second electronic device information gain, the maximum value of the local information gain is used for representing a maximum value of each feature in the first feature information set after splitting at each splitting threshold in the current node sample space, and the maximum value of the second electronic device information gain is obtained by decrypting the corresponding first fusion gradient; the dividing unit is configured to divide the sample space based on the characteristics corresponding to the target value of the information gain and the splitting threshold value in response to the target value of the information gain being the maximum value of the local information gain, and generate a sample space division result and a target fusion gradient corresponding to the sample space division result; the first updating unit is configured to update the model parameters of the first electronic device according to the target fusion gradient.
In some embodiments, the apparatus further comprises: a second transmitting unit configured to transmit, to the second electronic device, a random number code corresponding to the target value of the information gain in response to the target value of the information gain being a maximum value of the information gain of the second electronic device; and the second receiving unit is configured to receive the sample space division result sent by the second electronic device and the target fusion gradient corresponding to the sample space division result.
In some embodiments, the random number encoding in the first receiving unit is an encrypted random number encoding characterizing a split threshold for each feature and corresponding feature in the second set of feature information.
In some embodiments, the random number codes in the first receiving unit are generated by using a way of scrambling feature ordering based on each feature identifier in the second feature information set in the current node sample space and the splitting threshold of the corresponding feature, and the random number codes are different from each other.
In some embodiments, the apparatus further comprises: a third transmitting unit configured to transmit the sample space division results to the respective second electronic devices; and/or the second updating unit is configured to update the prediction result of the model based on the sample space division result when the depth threshold of the gradient lifting tree is reached.
In some embodiments, the apparatus further comprises: the generating unit is configured to generate a final model after training is completed when a tree threshold of the gradient lifting tree is reached, and send the final model to each second electronic device; and the third receiving unit is configured to receive the contribution degrees which are sent by the second electronic devices based on the final model and correspond to the characteristics in the second characteristic information set.
According to a fourth aspect of the present application, there is provided an electronic device comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described in any one of the implementations of the first aspect.
According to a fifth aspect of the present application, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions, wherein the computer instructions are configured to cause a computer to perform the method as described in any one of the implementations of the first aspect.
According to the technology of the application, when a tree threshold value of a gradient lifting tree is not reached and a depth threshold value of the gradient lifting tree is not reached, gradient data are sent to each second electronic device in a federal learning system, random number codes which are sent by each second electronic device based on the gradient data and represent splitting threshold values of each feature and corresponding features in a second feature information set and first fusion gradients corresponding to the random number codes are received, a target value of information gain is determined based on each first fusion gradient and the first feature information set, the target value of the information gain is the maximum value of the local information gain in response to the information gain, a sample space is divided based on the corresponding feature and the splitting threshold value of the target value of the information gain, a sample space division result and a target fusion gradient corresponding to the sample space division result are generated, and model parameters of the first electronic device are updated according to the target fusion gradient, the characteristic information is generated into a random number code and transmitted in a random code mode, so that the model training method with interpretability based on the federal learning is realized, and the problems that in the prior art, the characteristics cannot be disclosed due to the fact that tree model nodes and the characteristics are bound, and the sample characteristic information can be leaked once the characteristics are disclosed are solved. The feature information is generated into a random number code, the splitting feature of the second electronic equipment (namely, the Host party) node is hidden, the possibility that other parties guess the model information is prevented, the security of the feature data is protected, and the feature name is disclosed to the first electronic equipment (namely, the Guest party) by the Host party while the privacy information of the Host party is protected. In the process of searching the maximum information gain characteristic of the node, the original characteristic code is not exposed any more, but the random number of the characteristic is transmitted, so that the Guest party is difficult to identify whether the splitting characteristics of different nodes are consistent, the Guest party cannot count the splitting times of each characteristic, even if the Host party discloses the modulus contribution degree, the Guest party cannot find the node to which the corresponding characteristic belongs according to the same splitting times, the sample characteristic cannot be presumed through the quantity of leaf nodes, and the safety of characteristic data is guaranteed.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present application, nor do they limit the scope of the present application. Other features of the present application will become apparent from the following description.
Drawings
The drawings are included to provide a better understanding of the present solution and are not intended to limit the present application.
FIG. 1 is a schematic diagram of a first embodiment of a federated learning-based model training method in accordance with the present application;
FIG. 2 is a scenario diagram of a federated learning-based model training method that may implement an embodiment of the present application;
FIG. 3 is a schematic diagram of a second embodiment of a federated learning-based model training method in accordance with the present application;
FIG. 4 is a schematic structural diagram of one embodiment of a federated learning-based model training system in accordance with the present application;
FIG. 5 is a schematic diagram illustrating the structure of one embodiment of a federated learning-based model training apparatus in accordance with the present application;
FIG. 6 is a block diagram of an electronic device for implementing the federated learning-based model training method of an embodiment of the present application.
Detailed Description
The following description of the exemplary embodiments of the present application, taken in conjunction with the accompanying drawings, includes various details of the embodiments of the application for the understanding of the same, which are to be considered exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
FIG. 1 shows a schematic diagram 100 of a first embodiment of a federated learning-based model training method in accordance with the present application. The method is applied to a first electronic device of a federal learning system, the federal learning is developed for longitudinal federal learning, and the system can comprise: the system comprises at least one first electronic device (Guest party) and at least one second electronic device (Host party), wherein each user identification and a first characteristic information set of each user are stored in the first electronic device, a second characteristic information set of the same user as the first electronic device is stored in the second electronic device, and the first characteristic information set is different from the second characteristic information set. The first characteristic information Set and the second characteristic information Set are subjected to sample alignment before model training, wherein the sample alignment specifically means that Intersection parts of IDs of two parties are calculated through a PSI technology (Private Set interaction), complementary sets of the IDs of the two parties are not leaked while the Intersection parts of the IDs of the users are aligned, for example, different participants of a federated learning system complete sample matching of the same user ID under the condition that other user ID information is not leaked by adopting RSA and Hash algorithm. The model is constructed based on a Gradient lifting Tree algorithm of a Decision Tree model, a Guest party is used as a root node of the Decision Tree and has a trusted execution environment approved by a Host party, and the Gradient lifting Tree (Gradient lifting Decision Tree) algorithm can be generated based on multi-party expansion of the most representative Gradient lifting Tree XGboost algorithm. The model training method based on the federal learning comprises the following steps:
and step 101, when the tree threshold of the gradient lifting tree is not reached and the depth threshold of the gradient lifting tree is not reached, sending gradient data to each second electronic device in the federal learning system.
In this embodiment, when the executing subject (e.g. the cloud server or the terminal device) determines that the tree threshold of the gradient boosting tree is not reached, the tth tree is constructed from the first t-1 trees,until the upper tree threshold of the gradient lifting tree is reached. When the execution main body does not reach the depth threshold of the gradient lifting tree, node splitting is carried out until the depth threshold of the gradient lifting tree is reached, and the training tree f of the t-th tree is obtainedt(X). The enforcement agent may send the gradient data to each second electronic device in the federated learning system by way of a wired or wireless connection. The tree threshold and the depth threshold may be derived through local sync training. Gradient data is generated based on the predicted outcome of the constructed tree of the gradient-boosted tree. It should be noted that the above-mentioned wireless connection means may include, but is not limited to, 3G, 4G, 5G connection, WiFi connection, bluetooth connection, WiMAX connection, Zigbee connection, uwb (ultra wideband) connection, and other now known or later developed wireless connection means.
In some optional implementations of the present embodiment, the gradient data is used to characterize first and second order gradients of the respective samples; the gradient data is generated by encrypting the first and second order gradients using encryption techniques, including homomorphic encryption techniques. Iterative training based on second-order gradient is realized, and the safety of data is ensured by encrypting the transmission data.
The generation and transmission of gradient data specifically refer to: the first electronic device calculates a residual error according to the prediction results of the first t-1 trees, and calculates a first order gradient and a second order gradient of each sample. And the first electronic equipment performs Pallier addition homomorphic encryption on the first-order gradient and the second-order gradient of each sample and transmits the encrypted samples to the second electronic equipment.
And 102, receiving the random number codes representing the splitting threshold of each feature and the corresponding feature in the second feature information set and the first fusion gradient corresponding to each random number code, which are sent by each second electronic device based on the gradient data.
In this embodiment, the execution subject may receive, based on the gradient data, a random number code corresponding to each feature in the respective second feature information set and the splitting threshold of the corresponding feature and a first fusion gradient corresponding to each random number code, which are transmitted by the respective second electronic devices. The random number code may be represented by a number, letter, or symbol.
Here, the second electronic device calculates, for each local feature, a first fusion gradient corresponding to each feature and the binning threshold of the feature according to the binning information of the local feature and the binning threshold of the local feature in the current sample space. Meanwhile, the second electronic device randomly generates corresponding random number codes according to each feature in the second feature data set and the split threshold number of each feature, and sends each random number code and the first fusion gradient corresponding to each random number code to the first electronic device so that the first electronic device receives the random number codes.
In this embodiment, the execution subject may determine the target value of the information gain based on each first fusion gradient received in step 102 and the local first feature information set. The target value of the information gain refers to the maximum value among all information gain values, i.e. it can be determined whether the optimal split point position is located at the first electronic device or at some second electronic device. The target value of the information gain is obtained by selecting the maximum value of the local information gain and the maximum value of each second electronic device information gain, the maximum value of the local information gain is used for representing the maximum value of each feature in the first feature information set in the current node sample space in each information gain after splitting of each splitting threshold, and the maximum value of the second electronic device information gain is obtained by decrypting the corresponding first fusion gradient. And the judgment process of whether the node splitting is carried out is to calculate the score of the splitting point of the local splitting point and each random number of other parties according to the gradient information, stop the splitting of the current node if the score is smaller than a splitting threshold value, and otherwise carry out the node splitting.
And 104, in response to that the target value of the information gain is the maximum value of the local information gain, dividing the sample space based on the characteristics corresponding to the target value of the information gain and the splitting threshold value, and generating a sample space division result and a target fusion gradient corresponding to the sample space division result.
In this embodiment, when the execution subject determines that the target value of the information gain is the maximum value of the local information gain (i.e., the split point is located in the first electronic device), the left and right child node partitions are performed on the sample space based on the feature corresponding to the target value of the information gain and the split threshold, and the sample space partition result and the target fusion gradient corresponding to the sample space partition result are generated.
And 105, updating the model parameters of the first electronic equipment according to the target fusion gradient.
In this embodiment, the executing entity may update the model parameters of the first electronic device according to the target fusion gradient to complete one model training.
In some optional implementations of this embodiment, the method further includes: sending the sample space division result to each second electronic device; and/or updating the prediction result of the model based on the sample space division result when the depth threshold of the gradient lifting tree is reached. And the synchronization of model data is realized so as to complete the whole learning process of the federal learning.
In some optional implementations of this embodiment, the method further includes: when the tree threshold of the gradient lifting tree is reached, generating a final model after training is completed, and sending the final model to each second electronic device; and receiving contribution degrees which are sent by each second electronic device based on the final model and correspond to each feature in the second feature information set, wherein the contribution degrees are used for representing the number of times of appearance of the features. After the node splitting is completed, the Host side sends the feature names of the synchronous local model entering features and the contribution degree of each feature to the Guest side for the Guest side to evaluate the data quality of the model and the participant side, for example, the contribution degree can be used as a basis for measuring the importance of the model entering features to the model, and the Guest side can perform rationality analysis and evaluation on a modeling result. Because the feature information acquired by the Guest party is randomly generated by the Host party code each time, the Guest party cannot know whether the feature threshold values of each round of transmission are consistent or not and cannot know the split node corresponding to each feature, and thus data protection is realized.
It should be noted that the above gradient lifting tree algorithm, residual error, gradient value calculation, gradient fusion calculation, etc. are well-known technologies that are widely researched and applied at present, and are not described herein again.
With continued reference to fig. 2, the federal learning based model training method 200 of the present embodiment is implemented in an electronic device 201. When the electronic device 201 determines that the tree threshold of the gradient lifting tree is not reached and the depth threshold of the gradient lifting tree is not reached, the electronic device 201 first sends gradient data 202 to each second electronic device in the federal learning system, wherein the gradient data is generated based on a prediction result of a constructed tree of the gradient lifting tree, then receives random number codes representing split thresholds of each feature and corresponding features in second feature information sets sent by each second electronic device based on the gradient data and first fusion gradients 203 corresponding to each random number code, then the electronic device 201 determines a target value 204 of the information gain based on each first fusion gradient and each first feature information set, wherein the target value of the information gain is obtained by selecting the maximum value of the information gain of the local device and the maximum value of the information gain of each second electronic device, when the electronic device 201 determines that the target value of the information gain is the maximum value of the local information gain, the electronic device 201 divides the sample space based on the feature corresponding to the target value of the information gain and the splitting threshold to generate a sample space division result and a target fusion gradient 205 corresponding to the sample space division result, and finally, the electronic device 201 updates 206 the model parameter of the first electronic device according to the target fusion gradient.
The federate learning-based model training method provided in the foregoing embodiment of the present application sends gradient data to each second electronic device in the federate learning system when a tree threshold of a gradient-boosting tree is not reached and a depth threshold of the gradient-boosting tree is not reached, receives a random number code representing a split threshold of each feature and a corresponding feature in a second feature information set and a first fusion gradient corresponding to each random number code, which are sent by each second electronic device based on the gradient data, determines a target value of an information gain based on each first fusion gradient and a first feature information set, partitions a sample space based on the feature corresponding to the target value of the information gain and the split threshold in response to the target value of the information gain being a maximum value of the local information gain, and generates a sample space partitioning result and a target fusion gradient corresponding to the sample space partitioning result, according to the target fusion gradient, model parameters of the first electronic equipment are updated, feature information is generated into random number codes and transmitted in a random coding mode, the model training method with interpretability based on federal learning is achieved, and the problems that in the prior art, the features cannot be disclosed due to the fact that tree model nodes are bound with the features, and sample feature information may be leaked once the features are disclosed are solved. The feature information is generated into a random number code, the splitting feature of the second electronic equipment (namely, the Host party) node is hidden, the possibility that other parties guess the model information is prevented, the security of the feature data is protected, and the feature name is disclosed to the first electronic equipment (namely, the Guest party) by the Host party while the privacy information of the Host party is protected. In the process of searching the maximum information gain characteristic of the node, the original characteristic code is not exposed any more, but the random number of the characteristic is transmitted, so that the Guest party is difficult to identify whether the splitting characteristics of different nodes are consistent, the Guest party cannot count the splitting times of each characteristic, even if the Host party discloses the modulus contribution degree, the Guest party cannot find the node to which the corresponding characteristic belongs according to the same splitting times, the sample characteristic cannot be presumed through the quantity of leaf nodes, and the safety of characteristic data is guaranteed.
With further reference to FIG. 3, a diagram 300 of a second embodiment of a federated learning-based model training method is shown. The process of the method comprises the following steps:
and step 301, when the tree threshold of the gradient lifting tree is not reached and the depth threshold of the gradient lifting tree is not reached, sending gradient data to each second electronic device in the federal learning system.
In this embodiment, the execution subject may receive, based on the gradient data, a random number code corresponding to each feature in the respective second feature information set and the splitting threshold of the corresponding feature and a first fusion gradient corresponding to each random number code, which are transmitted by the respective second electronic devices. The random number code is an encrypted random number code characterizing a split threshold for each feature and corresponding feature in the second set of feature information. Random number codes are generated by means of disordering feature ordering based on each feature identifier in the second feature information set in the current node sample space and the splitting threshold of the corresponding feature, and the random number codes are different from each other.
And 304, responding to the target value of the information gain being the maximum value of the information gain of the second electronic device, and sending the random number code corresponding to the target value of the information gain to the second electronic device.
In this embodiment, when the execution subject determines that the target value of the information gain is the maximum value of the information gain of the second electronic device, the random number code corresponding to the target value of the information gain is transmitted to the second electronic device.
In this embodiment, the execution subject may receive the sample space division result sent by the second electronic device and the target fusion gradient corresponding to the sample space division result.
Here, the second electronic device may perform left and right sub-node division on the sample space based on the received random number code, generate a sample space division result and a target fusion gradient corresponding to the sample space division result, and transmit the sample space division result and the target fusion gradient to the first electronic device.
And step 306, updating the model parameters of the first electronic device according to the target fusion gradient.
In this embodiment, the specific operations of steps 301, 303, and 306 are substantially the same as the operations of steps 101, 103, and 105 in the embodiment shown in fig. 1, and are not described again here.
As can be seen from fig. 3, compared with the embodiment corresponding to fig. 1, the schematic diagram 300 of the model training method based on federal learning in this embodiment adopts a method that a target value in response to information gain is a maximum value of information gain of the second electronic device, sends a random number code corresponding to the target value of the information gain to the second electronic device, receives a sample space division result sent by the second electronic device and a target fusion gradient corresponding to the sample space division result, and updates the model parameters of the first electronic device according to the target fusion gradient, thereby implementing a model training process based on federal learning split by other parties. The feature information is generated into the random number codes in a disordered feature arrangement sequence, the random number codes are encrypted, the splitting features of the second electronic equipment (namely, Host party) nodes are hidden, the possibility that other parties guess model information is prevented, the security of the feature data is protected, the feature name is disclosed to the first electronic equipment (namely, Guest party) by the Host party, and meanwhile the privacy information of the Host party is protected.
With further reference to fig. 4, the present application provides a federal learning based model training system, as shown in fig. 4, comprising: a first electronic device 401 and at least one second electronic device 402, wherein the first electronic device is configured to execute any one of the above-mentioned federal learning based model training methods; the second electronic equipment is used for receiving the gradient data sent by the first electronic equipment in the federal learning system; generating random number codes corresponding to the splitting threshold values of each feature and the corresponding feature in the second feature information set and first fusion gradients corresponding to the random number codes based on the gradient data, and sending the random number codes and the first fusion gradients to the first electronic equipment; in response to receiving the random number code sent by the first electronic device, dividing the sample space based on the corresponding features of the random number code sent by the first electronic device and the splitting threshold value, and generating a sample space division result and a target fusion gradient corresponding to the sample space division result.
In the system, the second electronic device is further configured to send the sample space division result and the target fusion gradient corresponding to the sample space division result to the first electronic device.
In the system, the second electronic device is further configured to send, to the first electronic device, contribution degrees corresponding to respective features in the second feature information set in response to receiving the final model sent by the first electronic device, where the contribution degrees are obtained based on the final model.
The system realizes an interpretable model training system based on federal learning, random number codes representing characteristic information are sent to first electronic equipment through second electronic equipment, splitting characteristics of nodes of the second electronic equipment (namely a Host party) are hidden, possibility of other parties for estimating the model information is prevented, safety of characteristic data is protected, and privacy information of the Host party is protected while characteristic names are disclosed to the first electronic equipment (namely a Guest party) by the Host party. In the process of searching the maximum information gain characteristic of the node, the original characteristic code is not exposed any more, but the random number of the characteristic is transmitted, so that the Guest party is difficult to identify whether the splitting characteristics of different nodes are consistent, the Guest party cannot count the splitting times of each characteristic, even if the Host party discloses the modulus contribution degree, the Guest party cannot find the node to which the corresponding characteristic belongs according to the same splitting times, the sample characteristic cannot be presumed through the quantity of leaf nodes, and the safety of characteristic data is guaranteed.
With further reference to fig. 5, as an implementation of the methods shown in fig. 1 to 3, the present application provides an embodiment of a model training apparatus based on federal learning, where the embodiment of the apparatus corresponds to the embodiment of the method shown in fig. 1, and besides the features described below, the embodiment of the apparatus may further include the same or corresponding features as the embodiment of the method shown in fig. 1, and produce the same or corresponding effects as the embodiment of the method shown in fig. 1, and the apparatus may be applied to various electronic devices.
As shown in fig. 5, the federal learning based model training apparatus 500 of this embodiment is applied to a first electronic device in a federal learning system, where each user identifier and a first feature information set of each user are stored in the first electronic device, and the apparatus includes: the system comprises a first sending unit 501, a first receiving unit 502, a determining unit 503, a dividing unit 504 and a first updating unit 505, wherein the first sending unit is configured to send gradient data to each second electronic device in the federal learning system when a tree threshold of a gradient promotion tree is not reached and a depth threshold of the gradient promotion tree is not reached, wherein the gradient data is generated based on a prediction result of a constructed tree of the gradient promotion tree, and a second characteristic information set of the same user as that of the first electronic device is stored in the second electronic device; a first receiving unit configured to receive, based on the gradient data, random number codes representing splitting thresholds of each feature and a corresponding feature in the second feature information set and first fusion gradients corresponding to the respective random number codes, which are transmitted by the respective second electronic devices; a determining unit configured to determine a target value of the information gain based on each first fusion gradient and each first feature information set, wherein the target value of the information gain is obtained by selecting a maximum value of the local information gain and a maximum value of each second electronic device information gain, the maximum value of the local information gain is used for representing a maximum value of each feature in the first feature information set after splitting at each splitting threshold in the current node sample space, and the maximum value of the second electronic device information gain is obtained by decrypting the corresponding first fusion gradient; the dividing unit is configured to divide the sample space based on the characteristics corresponding to the target value of the information gain and the splitting threshold value in response to the target value of the information gain being the maximum value of the local information gain, and generate a sample space division result and a target fusion gradient corresponding to the sample space division result; the first updating unit is configured to update the model parameters of the first electronic device according to the target fusion gradient.
In this embodiment, specific processes of the first sending unit 501, the first receiving unit 502, the determining unit 503, the dividing unit 504, and the first updating unit 505 of the model training apparatus 500 based on federal learning and technical effects thereof may respectively refer to the related descriptions of step 101 to step 105 in the embodiment corresponding to fig. 1, and are not described herein again.
In some optional implementations of this embodiment, the apparatus further includes: a second transmitting unit configured to transmit, to the second electronic device, a random number code corresponding to the target value of the information gain in response to the target value of the information gain being a maximum value of the information gain of the second electronic device; and the second receiving unit is configured to receive the sample space division result sent by the second electronic device and the target fusion gradient corresponding to the sample space division result.
In some optional implementations of this embodiment, the random number encoding in the first receiving unit is an encrypted random number encoding that characterizes a split threshold for each feature and corresponding feature in the second feature information set.
In some optional implementations of this embodiment, the random number codes in the first receiving unit are generated by using a way of scrambling feature ordering based on each feature identifier in the second feature information set in the current node sample space and a splitting threshold of a corresponding feature, and the random number codes are different from each other.
In some optional implementations of this embodiment, the apparatus further includes: a third transmitting unit configured to transmit the sample space division results to the respective second electronic devices; and/or the second updating unit is configured to update the prediction result of the model based on the sample space division result when the depth threshold of the gradient lifting tree is reached.
In some optional implementations of this embodiment, the apparatus further includes: the generating unit is configured to generate a final model after training is completed when a tree threshold of the gradient lifting tree is reached, and send the final model to each second electronic device; and the third receiving unit is configured to receive the contribution degrees which are sent by the second electronic devices based on the final model and correspond to the characteristics in the second characteristic information set.
According to an embodiment of the present application, an electronic device and a readable storage medium are also provided.
Fig. 6 is a block diagram of an electronic device according to an embodiment of the present application, illustrating a method for model training based on federal learning. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application that are described and/or claimed herein.
As shown in fig. 6, the electronic apparatus includes: one or more processors 601, memory 602, and interfaces for connecting the various components, including a high-speed interface and a low-speed interface. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions for execution within the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input/output apparatus (such as a display device coupled to the interface). In other embodiments, multiple processors and/or multiple buses may be used, along with multiple memories and multiple memories, as desired. Also, multiple electronic devices may be connected, with each device providing portions of the necessary operations (e.g., as a server array, a group of blade servers, or a multi-processor system). In fig. 6, one processor 601 is taken as an example.
The memory 602 is a non-transitory computer readable storage medium as provided herein. Wherein the memory stores instructions executable by the at least one processor to cause the at least one processor to perform the method for model training based on federated learning provided herein. The non-transitory computer readable storage medium of the present application stores computer instructions for causing a computer to perform the federated learning-based model training method provided herein.
The memory 602, as a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules corresponding to the federal learning based model training method in the embodiments of the present application (e.g., the first transmitting unit 501, the first receiving unit 502, the determining unit 503, the dividing unit 504, and the first updating unit 505 shown in fig. 5). The processor 601 executes various functional applications of the server and data processing by running non-transitory software programs, instructions and modules stored in the memory 602, namely, implements the federate learning based model training method in the above method embodiments.
The memory 602 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the stored data area may store data created from use of the federal learning based model training electronic device, and the like. Further, the memory 602 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, memory 602 optionally includes memory located remotely from processor 601, and these remote memories may be connected to the federally learned model training electronic device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The electronic device of the model training method based on federal learning may further include: an input device 603 and an output device 604. The processor 601, the memory 602, the input device 603 and the output device 604 may be connected by a bus or other means, and fig. 6 illustrates the connection by a bus as an example.
The input device 603 may receive input numeric or character information and generate key signal inputs related to user settings and function control of the federally-learned model-trained electronic device, such as a touch screen, keypad, mouse, track pad, touch pad, pointer stick, one or more mouse buttons, track ball, joystick, or other input device. The output devices 604 may include a display device, auxiliary lighting devices (e.g., LEDs), and tactile feedback devices (e.g., vibrating motors), among others. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device can be a touch screen.
Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
These computer programs (also known as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented using high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
According to the technical scheme of the embodiment of the application, when the tree threshold of the gradient lifting tree is not reached and the depth threshold of the gradient lifting tree is not reached, gradient data are sent to each second electronic device in the federal learning system, random number codes which are sent by each second electronic device based on the gradient data and represent the splitting threshold of each feature and corresponding feature in the second feature information set and first fusion gradients corresponding to the random number codes are received, a target value of information gain is determined based on each first fusion gradient and the first feature information set, the target value of the information gain is the maximum value of the local information gain in response to the information gain, a sample space is divided based on the corresponding feature of the target value of the information gain and the splitting threshold, a sample space division result and a target fusion gradient corresponding to the sample space division result are generated, and according to the target fusion gradient, the model parameters of the first electronic device are updated, the feature information is generated into a random number code and transmitted in a random code mode, the model training method with interpretability based on the federal learning is achieved, and the problems that in the prior art, the feature cannot be disclosed due to the fact that tree model nodes are bound with the feature, and sample feature information may be leaked once the feature is disclosed are solved. The feature information is generated into a random number code, the splitting feature of the second electronic equipment (namely, the Host party) node is hidden, the possibility that other parties guess the model information is prevented, the security of the feature data is protected, and the feature name is disclosed to the first electronic equipment (namely, the Guest party) by the Host party while the privacy information of the Host party is protected. In the process of searching the maximum information gain characteristic of the node, the original characteristic code is not exposed any more, but the random number of the characteristic is transmitted, so that the Guest party is difficult to identify whether the splitting characteristics of different nodes are consistent, the Guest party cannot count the splitting times of each characteristic, even if the Host party discloses the modulus contribution degree, the Guest party cannot find the node to which the corresponding characteristic belongs according to the same splitting times, the sample characteristic cannot be presumed through the quantity of leaf nodes, and the safety of characteristic data is guaranteed.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present application may be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present application can be achieved, and the present invention is not limited herein.
The above-described embodiments should not be construed as limiting the scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (18)
1. A model training method based on federated learning is applied to first electronic equipment in a federated learning system, wherein first electronic equipment stores user identifications and first feature information sets of the users, and the model is constructed based on a gradient lifting tree algorithm, and the method comprises the following steps:
when a tree threshold value of the gradient promotion tree is not reached and a depth threshold value of the gradient promotion tree is not reached, sending gradient data to each second electronic device in the federated learning system, wherein the gradient data is generated based on a prediction result of a built tree of the gradient promotion tree, and a second feature information set of the same user in the first electronic device is stored in the second electronic device;
receiving random number codes representing the splitting threshold value of each feature and the corresponding feature in the second feature information set and first fusion gradients corresponding to the random number codes, which are sent by the second electronic devices based on the gradient data;
determining a target value of information gain based on each first fusion gradient and the first feature information set, wherein the target value of the information gain is obtained by selecting a maximum value of the local information gain and a maximum value of each second electronic device information gain, the maximum value of the local information gain is used for representing a maximum value of each feature in the first feature information set after splitting of each splitting threshold in a current node sample space, and the maximum value of the second electronic device information gain is obtained by decrypting the corresponding first fusion gradient;
in response to that the target value of the information gain is the maximum value of the local information gain, dividing a sample space based on the characteristics corresponding to the target value of the information gain and a splitting threshold value, and generating a sample space division result and a target fusion gradient corresponding to the sample space division result;
and updating the model parameters of the first electronic equipment according to the target fusion gradient.
2. The method according to claim 1, wherein after said determining a target value of an information gain based on the respective first fused gradient and the first set of feature information, further comprising:
responding to the target value of the information gain as the maximum value of the information gain of the second electronic equipment, and sending the random number code corresponding to the target value of the information gain to the second electronic equipment;
and receiving a sample space division result sent by the second electronic equipment and a target fusion gradient corresponding to the sample space division result.
3. The method of claim 1, wherein the random number encoding is an encrypted random number encoding characterizing a split threshold for each feature and corresponding feature in the second feature information set.
4. The method of claim 1, wherein the random number codes are generated by using a way of scrambling feature ordering based on a splitting threshold of each feature identifier and corresponding feature in the second feature information set in a current node sample space, and the random number codes are different from each other.
5. The method of claim 1, further comprising:
sending the sample space division result to each second electronic device; and/or the presence of a gas in the gas,
when the depth threshold of the gradient lifting tree is reached, updating the prediction result of the model based on the sample space division result.
6. The method of claim 5, further comprising:
when the tree threshold of the gradient lifting tree is reached, generating a final model after training is completed, and sending the final model to each second electronic device;
and receiving contribution degrees which are sent by the second electronic equipment based on the final model and correspond to the features in the second feature information set.
7. A federal learning based model training system, the system comprising: a first electronic device, wherein,
the first electronic device is configured to perform the federated learning-based model training method of any of claims 1-6.
8. The system of claim 7, wherein the system further comprises: at least one second electronic device;
the second electronic device is used for receiving the gradient data sent by the first electronic device in the federal learning system; generating a random number code corresponding to a splitting threshold value representing each feature and corresponding feature in the second feature information set and a first fusion gradient corresponding to each random number code based on the gradient data, and sending the random number code and the first fusion gradient to the first electronic device; in response to receiving the random number code sent by the first electronic device, dividing a sample space based on the corresponding features of the random number code sent by the first electronic device and a splitting threshold value, and generating a sample space division result and a target fusion gradient corresponding to the sample space division result.
9. The system of claim 7, wherein the second electronic device is further configured to send the sample space partitioning result and a target fusion gradient corresponding to the sample space partitioning result to the first electronic device.
10. The system of claim 7, wherein the second electronic device is further configured to send, to the first electronic device, a contribution degree corresponding to each feature in the second set of feature information in response to receiving a final model sent by the first electronic device, wherein the contribution degree is obtained based on the final model.
11. A model training device is applied to first electronic equipment in a federated learning system, wherein first electronic equipment stores user identifications and first feature information sets of the users, and a model is constructed based on a gradient lifting tree algorithm, and the device comprises:
a first sending unit, configured to send gradient data to each second electronic device in the federated learning system when a tree threshold of the gradient elevation tree is not reached and a depth threshold of the gradient elevation tree is not reached, where the gradient data is generated based on a prediction result of a constructed tree of the gradient elevation tree, and a second feature information set of the same user as that in the first electronic device is stored in the second electronic device;
a first receiving unit configured to receive, based on the gradient data, a random number code representing a splitting threshold of each feature and a corresponding feature in the second feature information set and a first fusion gradient corresponding to each random number code, which are transmitted by each of the second electronic devices;
a determining unit configured to determine a target value of the information gain based on each first fusion gradient and the first feature information set, wherein the target value of the information gain is obtained by selecting a maximum value of an information gain of a local side and a maximum value of an information gain of each second electronic device, the maximum value of the information gain of the local side is used for representing a maximum value of each feature in the first feature information set after splitting of each splitting threshold in a current node sample space, and the maximum value of the information gain of the second electronic device is obtained by decrypting the corresponding first fusion gradient;
the dividing unit is configured to divide a sample space based on the feature corresponding to the target value of the information gain and a splitting threshold value in response to the target value of the information gain being the maximum value of the local information gain, and generate a sample space division result and a target fusion gradient corresponding to the sample space division result;
a first updating unit configured to update the model parameters of the first electronic device according to the target fusion gradient.
12. The apparatus of claim 11, further comprising:
a second transmitting unit configured to transmit the random number code corresponding to the target value of the information gain to a second electronic device in response to the target value of the information gain being a maximum value of an information gain of the second electronic device;
the second receiving unit is configured to receive the sample space division result sent by the second electronic device and the target fusion gradient corresponding to the sample space division result.
13. The apparatus of claim 11, wherein the random number encoding in the first receiving unit is an encrypted random number encoding characterizing a split threshold for each feature and corresponding feature in the second set of feature information.
14. The apparatus according to claim 11, wherein the random number codes in the first receiving unit are generated by using a way of scrambling feature ordering based on a splitting threshold of each feature identifier and corresponding feature in the second feature information set in a current node sample space, and the random number codes are different from each other.
15. The apparatus of claim 11, further comprising:
a third transmitting unit configured to transmit the sample space division result to each of the second electronic devices; and/or the presence of a gas in the gas,
a second updating unit configured to update a prediction result of the model based on the sample space partitioning result when a depth threshold of the gradient lifting tree is reached.
16. The apparatus of claim 15, further comprising:
the generating unit is configured to generate a final model after training is completed when a tree threshold of the gradient lifting tree is reached, and send the final model to each second electronic device;
a third receiving unit configured to receive the contribution degrees corresponding to the respective features in the second feature information set, which are sent by the respective second electronic devices based on the final model.
17. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.
18. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110666784.1A CN113762328B (en) | 2021-06-16 | 2021-06-16 | Model training method, device, equipment and storage medium based on federal learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110666784.1A CN113762328B (en) | 2021-06-16 | 2021-06-16 | Model training method, device, equipment and storage medium based on federal learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113762328A true CN113762328A (en) | 2021-12-07 |
| CN113762328B CN113762328B (en) | 2023-09-26 |
Family
ID=78787462
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110666784.1A Active CN113762328B (en) | 2021-06-16 | 2021-06-16 | Model training method, device, equipment and storage medium based on federal learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113762328B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114118312A (en) * | 2022-01-29 | 2022-03-01 | 华控清交信息科技(北京)有限公司 | Vertical training method, device, electronic equipment and system for GBDT model |
| CN114529108A (en) * | 2022-04-22 | 2022-05-24 | 北京百度网讯科技有限公司 | Tree model based prediction method, apparatus, device, medium, and program product |
| CN115828022A (en) * | 2023-02-21 | 2023-03-21 | 中国电子科技集团公司第十五研究所 | Data identification method, federal training model, device and equipment |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110995737A (en) * | 2019-12-13 | 2020-04-10 | 支付宝(杭州)信息技术有限公司 | Gradient fusion method and device for federal learning and electronic equipment |
| CN111382875A (en) * | 2020-03-06 | 2020-07-07 | 深圳前海微众银行股份有限公司 | Federal model parameter determination method, device, equipment and storage medium |
| CN111582505A (en) * | 2020-05-14 | 2020-08-25 | 深圳前海微众银行股份有限公司 | Federal modeling method, device, equipment and computer readable storage medium |
| CN111598186A (en) * | 2020-06-05 | 2020-08-28 | 腾讯科技（深圳）有限公司 | Decision model training method, prediction method and device based on longitudinal federal learning |
| CN112288097A (en) * | 2020-10-29 | 2021-01-29 | 平安科技（深圳）有限公司 | Federal learning data processing method and device, computer equipment and storage medium |
| CN112364908A (en) * | 2020-11-05 | 2021-02-12 | 浙江大学 | Decision tree-oriented longitudinal federal learning method |
| US20210150269A1 (en) * | 2019-11-18 | 2021-05-20 | International Business Machines Corporation | Anonymizing data for preserving privacy during use for federated machine learning |
-
2021
- 2021-06-16 CN CN202110666784.1A patent/CN113762328B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20210150269A1 (en) * | 2019-11-18 | 2021-05-20 | International Business Machines Corporation | Anonymizing data for preserving privacy during use for federated machine learning |
| CN110995737A (en) * | 2019-12-13 | 2020-04-10 | 支付宝(杭州)信息技术有限公司 | Gradient fusion method and device for federal learning and electronic equipment |
| CN111382875A (en) * | 2020-03-06 | 2020-07-07 | 深圳前海微众银行股份有限公司 | Federal model parameter determination method, device, equipment and storage medium |
| CN111582505A (en) * | 2020-05-14 | 2020-08-25 | 深圳前海微众银行股份有限公司 | Federal modeling method, device, equipment and computer readable storage medium |
| CN111598186A (en) * | 2020-06-05 | 2020-08-28 | 腾讯科技（深圳）有限公司 | Decision model training method, prediction method and device based on longitudinal federal learning |
| CN112288097A (en) * | 2020-10-29 | 2021-01-29 | 平安科技（深圳）有限公司 | Federal learning data processing method and device, computer equipment and storage medium |
| CN112364908A (en) * | 2020-11-05 | 2021-02-12 | 浙江大学 | Decision tree-oriented longitudinal federal learning method |
Non-Patent Citations (1)
| Title |
|---|
| XIAOLIN CHEN，ET AL: "Fed-EINI: An Efficient and Interpretable Inference Framework for Decision Tree Ensembles in Federated Learning", ARXIV:2105.09540V1, pages 1 - 9 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114118312A (en) * | 2022-01-29 | 2022-03-01 | 华控清交信息科技(北京)有限公司 | Vertical training method, device, electronic equipment and system for GBDT model |
| CN114118312B (en) * | 2022-01-29 | 2022-05-13 | 华控清交信息科技(北京)有限公司 | Vertical training method, device, electronic equipment and system for GBDT model |
| CN114529108A (en) * | 2022-04-22 | 2022-05-24 | 北京百度网讯科技有限公司 | Tree model based prediction method, apparatus, device, medium, and program product |
| CN115828022A (en) * | 2023-02-21 | 2023-03-21 | 中国电子科技集团公司第十五研究所 | Data identification method, federal training model, device and equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113762328B (en) | 2023-09-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113762328A (en) | Model training method, device, equipment and storage medium based on federal learning | |
| US20230251898A1 (en) | Federated computing method, electronic device and storage medium | |
| CN112016632B (en) | Model joint training method, device, equipment and storage medium | |
| CN111708922A (en) | Model generation method and device for representing heterogeneous graph nodes | |
| US20210209417A1 (en) | Method and apparatus for generating shared encoder | |
| CN113505882B (en) | Data processing method based on federal neural network model, related equipment and medium | |
| US11048754B2 (en) | Using self-information scores for entities to determine whether to perform entity resolution | |
| CN111310204B (en) | Data processing method and device | |
| CN111934872B (en) | Key processing method, device, electronic equipment and storage medium | |
| US10642802B2 (en) | Identifying an entity associated with an online communication | |
| CN110909372A (en) | Data processing method, device, equipment and medium | |
| CN113722739B (en) | Gradient lifting tree model generation method and device, electronic equipment and storage medium | |
| CN111160552B (en) | News information recommendation processing method, device, equipment and computer storage medium | |
| CN113792883A (en) | Model training method, device, equipment and medium based on federal learning | |
| CN111767990A (en) | Neural network processing method and device | |
| CN111177479A (en) | Method and device for acquiring feature vectors of nodes in relational network graph | |
| CN116015960A (en) | Multi-node traffic confusion method, device, equipment and storage medium | |
| CN112559497B (en) | Data processing method, information transmission method, device and electronic equipment | |
| CN112598139B (en) | Category encoding method, category encoding device, category encoding apparatus, category encoding device, category encoding storage medium, and category encoding program product | |
| CN113435516B (en) | Data classification method and device | |
| CN112328807A (en) | Anti-cheating method, device, equipment and storage medium | |
| CN112598251A (en) | Processing method, device and equipment for classification model prediction result and storage medium | |
| CN113824546B (en) | Method and device for generating information | |
| CN114462535A (en) | Data classification method and device | |
| US20230186102A1 (en) | Training method and apparatus for neural network model, device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |