CN113792883A - Model training method, device, equipment and medium based on federal learning - Google Patents

Model training method, device, equipment and medium based on federal learning Download PDF

Info

Publication number
CN113792883A
CN113792883A CN202110233577.7A CN202110233577A CN113792883A CN 113792883 A CN113792883 A CN 113792883A CN 202110233577 A CN202110233577 A CN 202110233577A CN 113792883 A CN113792883 A CN 113792883A
Authority
CN
China
Prior art keywords
data
training
model
trained
label
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110233577.7A
Other languages
Chinese (zh)
Other versions
CN113792883B (en
Inventor
李怡欣
张一凡
王虎
黄志翔
彭南博
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jingdong Technology Holding Co Ltd
Original Assignee
Jingdong Technology Holding Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jingdong Technology Holding Co Ltd filed Critical Jingdong Technology Holding Co Ltd
Priority to CN202110233577.7A priority Critical patent/CN113792883B/en
Priority claimed from CN202110233577.7A external-priority patent/CN113792883B/en
Publication of CN113792883A publication Critical patent/CN113792883A/en
Application granted granted Critical
Publication of CN113792883B publication Critical patent/CN113792883B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/243—Classification techniques relating to the number of classes
    • G06F18/2431—Multiple classes

Abstract

The application discloses a method, a device, equipment and a medium for model training based on federal learning, wherein a specific implementation mode of the method comprises the following steps: acquiring a first training label set and a second training label from an expert experience label library, wherein the expert experience label library comprises a plurality of training labels set based on expert experience, and the first training label set comprises a plurality of different first training labels; for each data end, determining a first training label corresponding to the data end in a first training label set, and training a model in the data end based on the first training label corresponding to the data end; acquiring training data sent by each data terminal according to the trained model; and training the integrated die according to the training data and the second training label so that the trained integrated die integrates the data sent by each data end. This embodiment can promote the model training effect.

Description

Model training method, device, equipment and medium based on federal learning
Technical Field
The application relates to the technical field of computers, in particular to the field of artificial intelligence, and particularly relates to a method, a device, equipment and a medium for model training based on federal learning.
Background
Currently, data is particularly important for machine learning. By aggregating multi-party data to carry out federal learning, the effect of model training in machine learning can be improved.
The conventional federal learning-based model training method is usually based on a single label for training, and if the content of the label of the single label is unreasonable, the effect of model training is poor.
Disclosure of Invention
A method, apparatus, device and medium for model training based on federated learning are provided.
In a first aspect, an embodiment of the present disclosure provides a model training method based on federal learning, which is applied to a business end, and includes: acquiring a first training label set and a second training label from an expert experience label library, wherein the expert experience label library comprises a plurality of training labels set based on expert experience, and the first training label set comprises a plurality of different first training labels; for each data end, determining a first training label corresponding to the data end in a first training label set, and training a model in the data end based on the first training label corresponding to the data end; acquiring training data sent by each data terminal according to the trained model; and training the integrated die according to the training data and the second training label so that the trained integrated die integrates the data sent by each data end.
In some embodiments, training the model in the data end based on the first training label corresponding to the data end includes: acquiring compressed data output after the data compression processing is carried out on the model in the data end; and returning a corresponding optimization direction to the model in the data end based on the compressed data and the first training label corresponding to the data end, so that the model in the data end is trained according to the optimization direction.
In some embodiments, returning a corresponding optimization direction to the model in the data side based on the compressed data and the first training label corresponding to the data side includes: and in response to the fact that the model in the data end is determined to be not trained well, returning a corresponding optimization direction to the model in the data end based on the trained models of other data ends in each data end, the compressed data and the first training label corresponding to the data end.
In some embodiments, returning a corresponding optimization direction to the model in the data side based on the compressed data and the first training label corresponding to the data side includes: and in response to determining that the data terminal is not disconnected and no delay occurs, returning a corresponding optimization direction to the model in the data terminal based on the compressed data and the first training label corresponding to the data terminal.
In some embodiments, obtaining training data sent by each data terminal according to the trained model includes: determining a target data end which is not disconnected and has no delay in each data end; and acquiring training data sent by the target data terminal according to the trained model.
In some embodiments, returning a corresponding optimization direction to the model in the data side based on the compressed data and the first training label corresponding to the data side includes: determining at least one same group of data terminals corresponding to the data terminals; acquiring the compressed data of the same group sent by each data end of the same group; integrating the same group of data and the compressed data by using the cooperator corresponding to the data end to obtain integrated data; and returning a corresponding optimization direction to the model in the data end based on the integrated data and the first training label corresponding to the data end.
In some embodiments, training the matched die based on the training data and the second training labels comprises: selecting sample data from the training data, and executing the following training steps: inputting the sample data into the initial model to obtain an output result output by the initial model; determining a loss value of a loss function of the initial model based on the output result and the second training label; and in response to the loss value meeting the preset convergence condition, taking the initial model as a trained integrated model.
In some embodiments, the above method further comprises: and responding to the condition that the loss value does not meet the preset convergence condition, adjusting related parameters in the initial model, reselecting sample data in the training data, and continuing to execute the training step.
In a second aspect, an embodiment of the present disclosure provides a method for model training based on federal learning, which is applied to a data side, and includes: inputting original data into a model to be trained to obtain compressed data which is output by the model to be trained and is subjected to compression processing; sending the compressed data to a service end so that the service end returns to an optimization direction according to the compressed data; training a model to be trained according to the optimization direction to obtain a trained model; acquiring training data output by the trained model; and sending the training data to the business end so that the business end trains the integration model based on the training data.
In some embodiments, the above method further comprises: the compressed data is one-dimensional data.
In a third aspect, an embodiment of the present disclosure provides a model training apparatus based on federal learning, which is applied to a service end, and includes: the system comprises a label obtaining unit, a label obtaining unit and a label selecting unit, wherein the label obtaining unit is configured to obtain a first training label set and a second training label in an expert experience label library, the expert experience label library comprises a plurality of training labels set based on expert experience, and the first training label set comprises a plurality of different first training labels; the first model training unit is configured to determine, for each data end, a first training label corresponding to the data end in a first training label set, and train a model in the data end based on the first training label corresponding to the data end; the data acquisition unit is configured to acquire training data sent by each data terminal according to the trained model; and the second model training unit is configured to train the integrated die according to the training data and the second training labels, so that the trained integrated model integrates the data sent by each data end.
In some embodiments, the first model training unit is further configured to: acquiring compressed data output after the data compression processing is carried out on the model in the data end; and returning a corresponding optimization direction to the model in the data end based on the compressed data and the first training label corresponding to the data end, so that the model in the data end is trained according to the optimization direction.
In some embodiments, the first model training unit is further configured to: and in response to the fact that the model in the data end is determined to be not trained well, returning a corresponding optimization direction to the model in the data end based on the trained models of other data ends in each data end, the compressed data and the first training label corresponding to the data end.
In some embodiments, the first model training unit is further configured to: and in response to determining that the data terminal is not disconnected and no delay occurs, returning a corresponding optimization direction to the model in the data terminal based on the compressed data and the first training label corresponding to the data terminal.
In some embodiments, the data acquisition unit is further configured to: determining a target data end which is not disconnected and has no delay in each data end; and acquiring training data sent by the target data terminal according to the trained model.
In some embodiments, the first model training unit is further configured to: determining at least one same group of data terminals corresponding to the data terminals; acquiring the compressed data of the same group sent by each data end of the same group; integrating the same group of data and the compressed data by using the cooperator corresponding to the data end to obtain integrated data; and returning a corresponding optimization direction to the model in the data end based on the integrated data and the first training label corresponding to the data end.
In some embodiments, the second model training unit is further configured to: selecting sample data from the training data, and executing the following training steps: inputting the sample data into the initial model to obtain an output result output by the initial model; determining a loss value of a loss function of the initial model based on the output result and the second training label; and in response to the loss value meeting the preset convergence condition, taking the initial model as a trained integrated model.
In some embodiments, the second model training unit is further configured to: and responding to the condition that the loss value does not meet the preset convergence condition, adjusting related parameters in the initial model, reselecting sample data in the training data, and continuing to execute the training step.
In a fourth aspect, an embodiment of the present disclosure provides a model training device based on federal learning, which is applied to a data end, and includes: the data processing unit is configured to input original data into a model to be trained to obtain compressed data which is output by the model to be trained and is subjected to compression processing; the data sending unit is configured to send the compressed data to the service end so that the service end returns to the optimization direction according to the compressed data; the third model training unit is configured to train the model to be trained according to the optimization direction to obtain a trained model; a data acquisition unit configured to acquire training data output by the trained model; and the data sending unit is further configured to send the training data to the business end so that the business end trains the integration model based on the training data.
In some embodiments, the compressed data is one-dimensional data.
In a fifth aspect, an embodiment of the present disclosure provides a model training apparatus based on federal learning, including: one or more processors; storage means for storing one or more programs; when executed by one or more processors, cause the one or more processors to implement a method as in any one of the first or second aspects.
In a sixth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, on which a computer program is stored, characterized in that the program, when executed by a processor, implements the method according to any one of the first or second aspects.
According to the technology of the application, a model training method based on federal learning is provided, and model training can be performed by combining a plurality of training labels which are set based on expert experience in an expert experience label library. The process is integrated with expert experience, multi-dimensional training of the model is achieved based on the training labels, and the training effect of the model can be improved.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.
Drawings
The drawings are included to provide a better understanding of the present solution and are not intended to limit the present application. Wherein:
FIG. 1 is an exemplary system architecture diagram in which one embodiment of the present application may be applied;
FIG. 2 is a flow diagram of one embodiment of a federated learning-based model training method according to the present application;
FIG. 3 is a flow diagram of another embodiment of a federated learning-based model training method in accordance with the present application;
FIG. 4 is a schematic diagram of an application scenario of a federated learning-based model training method in accordance with the present application;
FIG. 5 is a schematic diagram illustrating the structure of one embodiment of a federated learning-based model training apparatus in accordance with the present application;
FIG. 6 is a schematic diagram illustrating the structure of one embodiment of a federated learning-based model training apparatus in accordance with the present application;
FIG. 7 is a block diagram of a Federal learning-based model training device for implementing the Federal learning-based model training method of an embodiment of the present application.
Detailed Description
The following description of the exemplary embodiments of the present application, taken in conjunction with the accompanying drawings, includes various details of the embodiments of the application for the understanding of the same, which are to be considered exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
FIG. 1 illustrates an exemplary system architecture 100 to which embodiments of the federated learning-based model training method or federated learning-based model training apparatus of the present application may be applied.
As shown in fig. 1, the system architecture 100 may include data terminals 101, 102, 103, a network 104, and a service terminal 105. The network 104 is used as a medium for providing communication links between the data terminals 101, 102, 103 and the service terminal 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
Data with different dimensions may be stored in the data terminals 101, 102, and 103, and a tag corresponding to the data may be stored in the service terminal 105. The system architecture 100 can be applied to a model training scenario based on federal learning, in which different dimensions of data stored in different data terminals need to be aggregated, a model needs to be trained together, and data privacy in each data terminal needs to be protected in the process. For example, the data stored in the data terminal 101 may be user characteristics of a user group in an a dimension, the data stored in the data terminal 102 may be user characteristics of the user group in a B dimension, the data stored in the data terminal 103 may be user characteristics of the user group in a C dimension, and the service terminal 105 stores a user tag corresponding to the user group. Under the scene of model training based on federal learning, the user characteristics of a user group in the A dimension, the B dimension and the C dimension can be aggregated, and the model is trained, so that the obtained model can output more accurate user classification information.
In this implementation, in the process of training the model by using the system architecture 100, the data terminals 101, 102, and 103 need to send compressed data after data compression processing to the service terminal 105 through the network 104, where the compressed data may be compressed data output by the model in each data terminal when the model in the data terminals 101, 102, and 103 is trained randomly. The service end 105 may receive the compressed data after data compression processing sent by the data ends 101, 102, and 103, respectively, determine an optimization direction of a model in the data end 101, an optimization direction of a model in the data end 102, and an optimization direction of a model in the data end 103 based on the compressed data and the plurality of first training labels, and send the optimization directions to the data ends 101, 102, and 103. After the data terminals 101, 102, and 103 receive the optimization direction, the model in the data terminals 101, 102, and 103 may be trained based on the optimization direction, so as to obtain the model trained in the data terminal 101, the model trained in the data terminal 102, and the model trained in the data terminal 103. At this point, training of the models in the data terminals 101, 102, 103 is completed.
Further, the data terminals 101, 102, and 103 may respectively send training data to the service terminal 105 based on the trained model, where the training data includes data output by the model trained in the data terminal 101, data output by the model trained in the data terminal 102, and data output by the model trained in the data terminal 103. The service end 105 may receive the training data, and train the integrated model based on the training data and a preset second training label to obtain a trained integrated model. The first training label and the second training label may be the same label or different labels, which is not limited in this embodiment.
The data terminals 101, 102, and 103 may be hardware or software. When the data terminals 101, 102, and 103 are hardware, they may be implemented as a distributed data terminal cluster formed by multiple data terminals, or may be implemented as a single data terminal. When the data terminal 101, 102, 103 is software, it may be implemented as a plurality of software or software modules (for example, for providing distributed services), or may be implemented as a single software or software module. And is not particularly limited herein.
It should be noted that the service end 105 may be hardware or software. When the service end 105 is hardware, it may be implemented as a distributed service end cluster formed by a plurality of service ends, or implemented as a single service end. When the service 105 is software, it may be implemented as multiple pieces of software or software modules (e.g., for providing distributed services), or as a single piece of software or software module. And is not particularly limited herein.
It should be noted that the model training method based on federal learning provided in the embodiment of the present application may be executed by the data terminals 101, 102, and 103, or may be executed by the service terminal 105. Accordingly, the model training apparatus based on federal learning may be disposed in the data terminals 101, 102, 103, or may be disposed in the service terminal 105. And is not particularly limited herein.
It should be understood that the number of data side, network and service side in fig. 1 is only illustrative. There may be any number of data, network and service ends, as desired for the implementation.
With continued reference to FIG. 2, a flow 200 of one embodiment of the federated learning-based model training method according to the present application is shown as applied to the business side. The model training method based on the federal learning comprises the following steps:
step 201, a first training label set and a second training label are obtained from an expert experience label library, where the expert experience label library includes a plurality of training labels set based on expert experience, and the first training label set includes a plurality of different first training labels.
In this embodiment, the data end may be a device storing data, and the service end may be a device storing a data tag. An expert experience label library is arranged in an executive body (such as a business end shown in figure 1) of the model training method based on the federal learning. When model training is carried out, required training labels can be selected from an expert experience label library according to training requirements and used as a first training label set and a second training label. The first training label in the first training label set may be the same as the second training label or different from the second training label, which is not limited in this embodiment.
It should be noted that the service end may be a device corresponding to a certain data end in the at least one data end, and the service end may also be another device independent of the at least one data end, which is not limited in this embodiment.
Step 202, for each data end, determining a first training label corresponding to the data end in a first training label set, and training a model in the data end based on the first training label corresponding to the data end.
In this embodiment, the executing entity can determine a first training label corresponding to each data end in the first training label set, and train the model in the data end based on the data sent to the executing entity by each data end and the first training label corresponding to the data end.
In some optional implementation manners of this embodiment, training the model in the data end based on the first training label corresponding to the data end includes: acquiring compressed data output after the data compression processing is carried out on the model in the data end; and returning a corresponding optimization direction to the model in the data end based on the compressed data and the first training label corresponding to the data end, so that the model in the data end is trained according to the optimization direction.
In this implementation manner, the execution main body can receive compressed data that is output after data compression processing and is sent by at least one data end (such as the data end shown in fig. 1), respectively, and return a corresponding optimization direction to each data end according to a first training label corresponding to each data end. Wherein different first training labels may correspond to different training directions. And different data terminals can store data with different dimensions. Each data end has a local model, and the model may be a deep learning model such as a deep neural network, or a traditional machine learning model such as a decision tree, which is not limited in this embodiment. The data sent by the data end is data output by the model of each data end in at least one data end.
Further, the optimization direction is used to describe an error value between the integrated data and a training target, for example, the training target of the coordinator is to make a value output by the model at the data end be a target value, after receiving data sent by the model at the data end, the data may be compared with the target value to determine an error value, for example, the data is the target value plus three, and the error value is determined to be positive three, where the corresponding model optimization direction is a direction that makes the value output by the model at the data end face the direction of minus three. Each of the at least one collaborator may integrate the compressed data respectively sent by the at least one data terminal to obtain integrated data corresponding to each collaborator. And determining the optimization direction of the model of each data end in the at least one data end based on the first training label corresponding to each collaborator and the integrated data corresponding to each collaborator. Each of the coordinators may correspond to the first training labels of different dimensions, so as to implement multidimensional model training on the model of each data end of the at least one data end. At this time, each data end in the at least one data end can receive the model optimization directions returned by the plurality of coordinators, and model training is performed based on the plurality of model optimization directions, so that the accuracy of model training can be improved.
In some optional implementation manners of this embodiment, returning a corresponding optimization direction to the model in the data end based on the compressed data and the first training label corresponding to the data end includes: and in response to the fact that the model in the data end is determined to be not trained well, returning a corresponding optimization direction to the model in the data end based on the trained models of other data ends in each data end, the compressed data and the first training label corresponding to the data end.
In this implementation manner, when model training is performed on a model in at least one data end, models of each data end in at least one data end are not necessarily completely and synchronously trained, and there may be a case where some models are trained and some models are not trained, and for this case, in response to that the model in the data end is not trained, the trained models in other data ends are used to assist in training the model in the data end. That is, based on the trained models in the other data terminals, the compressed data sent by the data terminal, and the first training label corresponding to the data terminal, the corresponding optimization direction is returned to the model in the data terminal. Or before model training is performed on the model in at least one data end, the priority of each model for training may be set, when training is performed, the model with the high priority is trained first to obtain a trained model, and when training the model with the low priority, the optimization direction of the untrained model may be determined by combining the trained model with the high priority, the compressed data sent by the data end, and the first training label set. This process can use the trained model to assist in training the untrained model, which can increase the training speed of the untrained model. The method is characterized in that the trained model is used for assisting in training the untrained model and can be realized based on a transfer learning technology, wherein the transfer learning is a method for deep learning by taking a pre-trained model as a starting point of a new model in a computer vision task and a natural language processing task.
In some optional implementation manners of this embodiment, returning a corresponding optimization direction to the model in the data end based on the compressed data and the first training label corresponding to the data end includes: and in response to determining that the data terminal is not disconnected and no delay occurs, returning a corresponding optimization direction to the model in the data terminal based on the compressed data and the first training label corresponding to the data terminal.
In this implementation manner, in the process of model training, the service end integrates compressed data respectively sent by each data end in at least one data end, and if a drop or delay occurs in one or more data ends at this time, it can be determined that a model in the data end operates abnormally, and at this time, the service end may not integrate compressed data sent by the model in the data end in this round of training, and the model in the data end is not trained in this round of training, but based on compressed data sent by models in other data ends that operate normally and the first training label set, the optimization direction of the model that operates normally in at least one data end is determined. That is, the corresponding optimized direction is not returned to the data end with the drop or delay in the training round. The process can decouple at least one data end participating in training to a greater extent, and the robustness of model training is improved.
Optionally, after the service end determines the optimization direction of the current round of training of each data end, the optimization direction may be sent to the corresponding data end, so that each data end in the at least one data end trains the model based on the corresponding model optimization direction. And repeatedly executing the step 202 until the model in the data end is converged to obtain the trained model in each data end in at least one data end. The process completes the training of the local model in the at least one data terminal, and then the integrated model in the service terminal can be trained based on the trained model in the at least one data terminal.
In some optional implementation manners of this embodiment, returning a corresponding optimization direction to the model in the data end based on the compressed data and the first training label corresponding to the data end includes: determining at least one same group of data terminals corresponding to the data terminals; acquiring the compressed data of the same group sent by each data end of the same group; integrating the same group of data and the compressed data by using the cooperator corresponding to the data end to obtain integrated data; and returning a corresponding optimization direction to the model in the data end based on the integrated data and the first training label corresponding to the data end.
In this implementation, the service end may include at least one coordinator, and the coordinator has a corresponding first training tag. Each of the at least one cooperators is configured to integrate compressed data sent by each of the at least one data end. Each of the synergists has a corresponding first training label, and the number of the first training labels corresponding to each of the synergists may be one or more than one, which is not limited in this embodiment. The first training labels in different synergists may correspond to training targets with different dimensions, for example, the training targets of different synergists may be classifications for different attributes, the training target of each synergist is to classify the attribute corresponding to the synergist, and the attributes of different synergists are different. For example, assuming that there are 3 data terminals and the service terminal has 3 coordinators, each of the coordinators may integrate the compressed data sent by the 3 data terminals. And each coordinator also has a corresponding first training label, and at the moment, the compressed data sent by the data end is integrated by using each coordinator, so that the model in the data end can be trained based on different training labels. Because the first training labels corresponding to different coordinators can be training labels with different dimensions, compressed data sent by a data end is integrated by adopting the coordinators, and multidimensional training of a model in the data end can be realized. Further, each of the coordinators may be responsible for integrating compressed data sent by a group of data terminals, and the at least one same group of data terminals and the data terminal may be data terminals for which the same coordinator is responsible for integration.
In some optional implementation manners of this embodiment, a manner in which each data terminal generates compressed data after data compression based on a local model may specifically be: and (4) randomly training the model at the data end, and inputting random data into the model so that the model outputs compressed data. The compressed data may be data obtained by performing data compression on random data. The information entropy of the compressed data can be improved by using the model of the data end to compress the data, and the privacy of data transmission is ensured. Moreover, the encryption and decryption algorithm is not needed in the process, so that the complexity of maintaining data security is reduced.
And step 203, acquiring training data sent by each data terminal according to the trained model.
In this embodiment, after the model in the data end satisfies the convergence condition and the trained model is obtained, each data end may send training data output by the trained model to the service end, and the service end may receive training data sent by at least one data end based on the trained model.
In some optional implementation manners of this embodiment, obtaining training data sent by each data end according to the trained model includes: determining a target data end which is not disconnected and has no delay in each data end; and acquiring training data sent by the target data terminal according to the trained model.
In this implementation, after the model in the data end satisfies the convergence condition and a trained model is obtained, if it is determined that a certain data end has a dropped connection or a delay, the data end is not used when the entire die type is trained, and therefore, only training data sent by a target data end that has no dropped connection and no delay needs to be obtained.
And step 204, training the integrated model according to the training data and the second training labels, so that the trained integrated model integrates the data sent by each data end.
In this embodiment, the service end may train the integration model based on the training data and the second training label.
In some optional implementations of this embodiment, training the matched model according to the training data and the second training label includes: selecting sample data from the training data, and executing the following training steps: inputting the sample data into the initial model to obtain an output result output by the initial model; determining a loss value of a loss function of the initial model based on the output result and the second training label; and in response to the loss value meeting the preset convergence condition, taking the initial model as a trained integrated model.
The initial model is an initial integrated model, sample data can be selected from training data to serve as input data of the initial model, an output result output by the initial model is obtained, and a loss value of a loss function of the initial model is determined based on the output result and a second training label, wherein a loss value calculation formula is as follows:
L(ρi，fi，Y)＝arg min((Y-F(ρi，fi)))
wherein L (ρ)i，fiY) denotes the loss value of the loss function, piRefer to the hyper-parameter, fi refers to the training data, and Y refers to the second training label. Wherein the hyperparameter rhoiFor adjusting the importance of each fi. F means for integrating each FiThe integration manner of (a) may specifically be a summation manner, and the like, which is not limited in this embodiment. And at each fiWith corresponding hyperparameters ρiIn the case of (3), F may be each of FiCorresponding hyperparameter rhoiAnd carrying out weighted summation. The predicted value of the integrated model in the training round can be obtained through F, and Y is usedAnd subtracting the predicted value to obtain the loss value of the loss function of the training of the current round.
And substituting the training data, the hyper-parameters and the training labels of each round of training into the loss value calculation formula to calculate the loss value of the loss function of the initial model. And repeating multiple rounds of training until the loss value meets the preset convergence condition, and determining the initial model with the loss value meeting the preset convergence condition as the integrated model.
In some optional implementations of this embodiment, the following steps may also be performed: and responding to the loss value not meeting the preset convergence condition, adjusting the related parameters in the initial model, reselecting sample data in the training data, and continuing to execute the training step.
If the loss value does not meet the preset convergence condition, relevant parameters in the initial model can be adjusted, sample data is reselected from the training data, multiple rounds of training are started, and the training step is continuously executed.
The federate learning-based model training method provided by the above embodiment of the application can be used for performing model training by combining a plurality of training labels which are set based on expert experience in an expert experience label library. The process is integrated with expert experience, multi-dimensional training of the model is achieved based on the training labels, and the training effect of the model can be improved. In addition, when the model in the data end is trained, each round of training only needs the service end to return to the model optimization direction, namely, only needs one time of information interaction, and when the model is integrated in the service end, each round of training also only needs to acquire the data output by the trained model in the data end, and the model training can be completed through one time of information interaction. By adopting the model training method, the communication times between the service end and the data end can be reduced, so that the model training complexity is reduced.
With continued reference to FIG. 3, a flow 300 of one embodiment of the federated learning-based model training method according to the present application is shown as applied to the data side. The model training method based on the federal learning comprises the following steps:
step 301, inputting the original data into the model to be trained to obtain compressed data output by the model to be trained after compression processing.
In this embodiment, the original data may be input data input to the model when the model is randomly trained. The original data may be high-dimensional data, and after the original data is input into the model, the model may perform data compression on the original data to obtain compressed data output by the model after compression processing. The compressed data may be one-dimensional data obtained by performing data compression on high-dimensional data. The one-dimensional compressed data is sent to the service end, so that the service end is difficult to deduce information of the original data from the compressed data, and the data security is improved. Optionally, data compression may be performed based on a deep learning model or a conventional machine learning model, which is not limited in this embodiment. The compressed data may be one-dimensional data.
Step 302, sending the compressed data to the service end, so that the service end returns to the optimization direction according to the compressed data.
In this embodiment, since the number of the data ends may be greater than one, in the case that the data end fails to send the compressed data to the service end in the current training round due to data end drop or data end delay, the service end may ignore the data end, integrate the compressed data sent by other data ends, and return the optimization direction to the models in other data ends in the current training round.
And 303, training the model to be trained according to the optimized direction to obtain the trained model.
In this embodiment, the data end may adjust parameters of a model in the data end based on the optimization direction returned by the service end, and output compressed data by using the model after parameter adjustment until the model in the data end converges to obtain a trained model.
And step 304, acquiring training data output by the trained model.
In this embodiment, after completing the model training in the service end, data may be input to the trained model, where the data may be pre-stored data, and after inputting the data to the trained model, training data output by the trained model may be obtained. The training data may be data after data compression.
Step 305, sending the training data to the business end, so that the business end trains the integration model based on the training data.
In this embodiment, after the data end trains the local model, data may be input to the trained local model to obtain training data output by the trained model, and the training data may be used as input data for the service end to train the integrated model, so that the service end realizes training of the integrated model based on the training data.
With continued reference to fig. 4, fig. 4 is a schematic diagram of an application scenario of model training based on federated learning according to the present embodiment. In the application scenario of fig. 4, there are models 401 in the a-group data side, models 402 in the B-group data side, and models 403 in the C-group data side. Each model 401 in the a-group data end may send compressed data to a coordinator D404 in the service end, each model 402 in the B-group data end may send compressed data to a coordinator E405 in the service end, and each model 403 in the C-group data end may send compressed data to a coordinator F406 in the service end. It should be noted that each model 401 in the a-group data side may be a model in each of the plurality of data sides. Similarly, each model 402 in the B-group data terminal and each model 403 in the C-group data terminal may also be a model in each of the plurality of data terminals. The coordinator D404 may receive compressed data sent by each model 401 in the a group of data ends, determine an optimization direction of each model 401 in the a group of data ends based on a first training label corresponding to the coordinator D404, and return the optimization direction to a corresponding model in the data end, so that the model in the data end is trained based on the optimization direction, and finally a trained model is obtained. Similarly, the cooperator E405 and the cooperator F406 may also respectively return a corresponding optimization direction to each model 402 in the B group data end and each model 403 in the C group data end based on the received compressed data, so that the models in the data end are trained based on the optimization directions, and finally the trained models are obtained. Each of the synergists corresponds to a corresponding first training label, and the first training labels of different synergists may be labels of different dimensions. Meanwhile, the coordinator D404, the coordinator E405, and the coordinator F406 may train the models of the coordinator itself based on the data integration result of the training of the current round and the first training label in the process of integrating the models in the corresponding group of data ends to determine the optimization directions of the models, and finally obtain the trained coordinator.
After A, B, C groups of data ends are trained, training data can be obtained based on the trained models, and the training data are sent to the cooperator D404, the cooperator E405, and the cooperator F406, respectively, so that the cooperator D404, the cooperator E405, and the cooperator F406 use the training data as input data of the integrated model 407, and the integrated model 407 is trained based on the input data of the integrated model 407 and a preset second training label, so that a trained integrated model is obtained. In this process, a first training label and a second training label fused with expert experience may be preset, and the first training label and the second training label may be the same or different. The expert experience label library comprises a plurality of preset training labels fusing expert experiences, the first training label set and the second training labels are obtained from the expert experience label library, the training labels fusing the expert experiences can be selected, and the training labels are used for model training, so that the effect is better. Model and integration model in the data end are trained respectively based on the first training label set and the second training label, and multidimensional model training can be achieved based on different training labels. In addition, A, B, C each model in group data end can compress high-dimensional data into one-dimensional data and transmit to the service end, has guaranteed data security to need not to realize encryption transmission based on cryptography, reduced the model training complexity. Compared with the frequent communication caused by encryption transmission based on cryptography, the data compression is adopted to replace encryption processing, the frequent communication brought by the encryption and decryption process can be reduced, and the communication frequency of a data end and a service end is reduced, so that the coupling degree of each party participating in model training is reduced, and each party participating in model training can be decoupled to a certain degree.
With further reference to fig. 5, as an implementation of the method shown in the above-mentioned figures, the present application provides an embodiment of a model training apparatus based on federal learning, where the embodiment of the apparatus corresponds to the embodiment of the method shown in fig. 2, and the apparatus may be applied to various electronic devices.
As shown in fig. 5, the model training apparatus 500 based on federal learning of the present embodiment includes: a label obtaining unit 501, a first model training unit 502, a data obtaining unit 503, and a second model training unit 504.
A label obtaining unit 501 configured to obtain a first training label set and a second training label in an expert experience label library, where the expert experience label library includes a plurality of training labels set based on expert experience, and the first training label set includes a plurality of different first training labels; a first model training unit 502 configured to, for each data end, determine a first training label corresponding to the data end in a first training label set, and train a model in the data end based on the first training label corresponding to the data end; a data obtaining unit 503 configured to obtain training data sent by each data terminal according to the trained model; the second model training unit 504 is configured to train the integrated model according to the training data and the second training labels, so that the trained integrated model integrates the data sent by each data end.
In some optional implementations of this embodiment, the first model training unit 502 is further configured to: acquiring compressed data output after the data compression processing is carried out on the model in the data end; and returning a corresponding optimization direction to the model in the data end based on the compressed data and the first training label corresponding to the data end, so that the model in the data end is trained according to the optimization direction.
In some optional implementations of this embodiment, the first model training unit 502 is further configured to: and in response to the fact that the model in the data end is determined to be not trained well, returning a corresponding optimization direction to the model in the data end based on the trained models of other data ends in each data end, the compressed data and the first training label corresponding to the data end.
In some optional implementations of this embodiment, the first model training unit 502 is further configured to: and in response to determining that the data terminal is not disconnected and no delay occurs, returning a corresponding optimization direction to the model in the data terminal based on the compressed data and the first training label corresponding to the data terminal.
In some optional implementations of this embodiment, the data obtaining unit is further configured to: determining a target data end which is not disconnected and has no delay in each data end; and acquiring training data sent by the target data terminal according to the trained model.
In some optional implementations of this embodiment, the first model training unit 502 is further configured to: determining at least one same group of data terminals corresponding to the data terminals; acquiring the compressed data of the same group sent by each data end of the same group; integrating the same group of data and the compressed data by using the cooperator corresponding to the data end to obtain integrated data; and returning a corresponding optimization direction to the model in the data end based on the integrated data and the first training label corresponding to the data end.
In some optional implementations of this embodiment, the second model training unit 504 is further configured to: selecting sample data from the training data, and executing the following training steps: inputting the sample data into the initial model to obtain an output result output by the initial model; determining a loss value of a loss function of the initial model based on the output result and the second training label; and in response to the loss value meeting the preset convergence condition, taking the initial model as a trained integrated model.
In some optional implementations of this embodiment, the second model training unit 504 is further configured to: and responding to the condition that the loss value does not meet the preset convergence condition, adjusting related parameters in the initial model, reselecting sample data in the training data, and continuing to execute the training step.
It should be understood that the units 501 to 504 described in the federal learning-based model training device 500 correspond to the respective steps in the method described with reference to fig. 2, respectively. Thus, the operations and features described above for the federated learning-based model training method are equally applicable to the apparatus 500 and the elements contained therein and will not be described in further detail herein.
With further reference to fig. 6, as an implementation of the method shown in the above-mentioned figures, the present application provides an embodiment of a model training apparatus based on federal learning, where the embodiment of the apparatus corresponds to the embodiment of the method shown in fig. 3, and the apparatus may be applied to various electronic devices.
As shown in fig. 6, the model training apparatus 600 based on federal learning of the present embodiment includes: a data processing unit 601, a data transmission unit 602, a third model training unit 603, a data acquisition unit 604, and a data transmission unit 605.
A data processing unit 601 configured to input raw data into a model to be trained, and obtain compressed data output by the model to be trained after compression processing; a data sending unit 602 configured to send the compressed data to the service end, so that the service end returns an optimization direction according to the compressed data; a third model training unit 603 configured to train a model to be trained according to the optimization direction to obtain a trained model; a data acquisition unit 604 configured to acquire training data output by the trained model; the data sending unit 605 is further configured to send the training data to the service end, so that the service end trains the integration model based on the training data.
In some alternative implementations of the present embodiment, the compressed data is one-dimensional data.
According to the embodiment of the application, the application also provides a model training device based on the federal learning and a readable storage medium.
Referring now to fig. 7, shown is a block diagram of a federated learning-based model training apparatus (e.g., the business side or the data side of fig. 1) 700 in accordance with an embodiment of the present application. The data side/service side shown in fig. 7 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure. The federal learning based model training device is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application that are described and/or claimed herein.
As shown in fig. 7, the model training apparatus based on federal learning includes: one or more processors 701, a memory 702, and interfaces for connecting the various components, including a high-speed interface and a low-speed interface. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions for execution within the federal learning based model training device, including instructions stored in or on a memory to display graphical information of a GUI on an external input/output device (such as a display device coupled to an interface). In other embodiments, multiple processors and/or multiple buses may be used, along with multiple memories and multiple memories, as desired. Also, multiple federated learning-based model training devices may be connected, each providing some of the necessary operations (e.g., as a server array, a set of blade servers, or a multi-processor system). In fig. 7, one processor 701 is taken as an example.
The memory 702 is a non-transitory computer readable storage medium as provided herein. Wherein the memory stores instructions executable by the at least one processor to cause the at least one processor to perform the method for model training based on federated learning provided herein. The non-transitory computer readable storage medium of the present application stores computer instructions for causing a computer to perform the federated learning-based model training method provided herein.
The memory 702 serves as a non-transitory computer readable storage medium, and may be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules corresponding to the federal learning based model training method in the embodiment of the present application (for example, the tag acquisition unit 501, the first model training unit 502, the data acquisition unit 503, and the second model training unit 504 shown in fig. 5, or the data processing unit 601, the data transmission unit 602, the third model training unit 603, the data acquisition unit 604, and the data transmission unit 605 shown in fig. 6). The processor 701 executes various functional applications and data processing of the federal learning based model training device by executing non-transitory software programs, instructions and modules stored in the memory 702, that is, implements the federal learning based model training method in the above method embodiments.
The memory 702 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the stored data area may store data created from use of the federal learning-based model training device, and the like. Further, the memory 702 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory 702 may optionally include memory located remotely from the processor 701, which may be connected through a network to an electronic device executing the software for storing data. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The model training device based on federal learning may further include: an input device 703 and an output device 704. The processor 701, the memory 702, the input device 703 and the output device 704 may be connected by a bus or other means, and fig. 7 illustrates an example of a connection by a bus.
The input device 703 may receive input numeric or character information and generate key signal inputs related to performing user settings and function control of the electronic apparatus for storing data, such as an input device like a touch screen, a keypad, a mouse, a track pad, a touch pad, a pointing stick, one or more mouse buttons, a track ball, a joystick, etc. The output devices 704 may include a display device, auxiliary lighting devices (e.g., LEDs), and tactile feedback devices (e.g., vibrating motors), among others. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device can be a touch screen.
Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
These computer programs (also known as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented using high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
According to the technical scheme of the embodiment of the application, the model training method based on the federal learning is provided, and the model training effect can be improved.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present application may be executed in parallel, sequentially, or in different orders, and the present invention is not limited thereto as long as the desired results of the technical solutions disclosed in the present application can be achieved.
The above-described embodiments should not be construed as limiting the scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims (22)

1. A model training method based on federal learning is applied to a business end, and the method comprises the following steps:
acquiring a first training label set and a second training label from an expert experience label library, wherein the expert experience label library comprises a plurality of training labels set based on expert experience, and the first training label set comprises a plurality of different first training labels;
for each data end, determining a first training label corresponding to the data end in the first training label set, and training a model in the data end based on the first training label corresponding to the data end;
acquiring training data sent by each data terminal according to the trained model;
and training the integrated die according to the training data and the second training labels so that the trained integrated die integrates the data sent by each data end.
2. The method of claim 1, wherein training the model in the data terminal based on the first training label corresponding to the data terminal comprises:
acquiring compressed data output after the data compression processing is carried out on the model in the data end;
and returning a corresponding optimization direction to the model in the data end based on the compressed data and the first training label corresponding to the data end, so that the model in the data end is trained according to the optimization direction.
3. The method of claim 2, wherein returning the corresponding optimization direction to the model in the data side based on the compressed data and the first training label corresponding to the data side comprises:
and in response to the fact that the model in the data end is determined to be not trained well, returning a corresponding optimization direction to the model in the data end based on the trained models of other data ends in each data end, the compressed data and the first training label corresponding to the data end.
4. The method of claim 2, wherein returning the corresponding optimization direction to the model in the data side based on the compressed data and the first training label corresponding to the data side comprises:
and in response to determining that the data end is not disconnected and has no delay, returning a corresponding optimization direction to the model in the data end based on the compressed data and the first training label corresponding to the data end.
5. The method according to claim 1, wherein the obtaining of the training data sent by each data terminal according to the trained model comprises:
determining a target data end which is not disconnected and has no delay in each data end;
and acquiring training data sent by the target data terminal according to the trained model.
6. The method of claim 2, wherein returning the corresponding optimization direction to the model in the data side based on the compressed data and the first training label corresponding to the data side comprises:
determining at least one same group of data terminals corresponding to the data terminals;
acquiring the compressed data of the same group sent by each data end of the same group;
integrating the same group of data and the compressed data by using a coordinator corresponding to the data end to obtain integrated data;
and returning a corresponding optimization direction to the model in the data end based on the integrated data and the first training label corresponding to the data end.
7. The method of any of claims 1 to 6, wherein said training a matched mold based on said training data and said second training label comprises:
selecting sample data from the training data, and executing the following training steps: inputting the sample data into an initial model to obtain an output result output by the initial model; determining a loss value of a loss function of the initial model based on the output result and the second training label; and in response to the loss value meeting a preset convergence condition, taking the initial model as a trained integrated model.
8. The method of claim 7, wherein the method further comprises:
and responding to the loss value not meeting the preset convergence condition, adjusting related parameters in the initial model, reselecting sample data in the training data, and continuing to execute the training step.
9. A model training method based on federated learning is applied to a data end, and the method comprises the following steps:
inputting original data into a model to be trained to obtain compressed data which is output by the model to be trained and is subjected to compression processing;
sending the compressed data to a service end so that the service end returns to an optimization direction according to the compressed data;
training the model to be trained according to the optimization direction to obtain a trained model;
acquiring training data output by the trained model;
and sending the training data to the business terminal so that the business terminal trains an integration model based on the training data.
10. The method of claim 9, wherein the compressed data is one-dimensional data.
11. A model training device based on federal learning is applied to a business end, and the device comprises:
a label obtaining unit configured to obtain a first training label set and a second training label in an expert experience label library, the expert experience label library including a plurality of training labels set based on expert experience, the first training label set including a plurality of different first training labels;
a first model training unit configured to determine, for each data end, a first training label corresponding to the data end in the first training label set, and train a model in the data end based on the first training label corresponding to the data end;
the data acquisition unit is configured to acquire training data sent by each data terminal according to the trained model;
and the second model training unit is configured to train the integrated die according to the training data and the second training labels, so that the trained integrated model integrates the data sent by each data end.
12. The apparatus of claim 11, wherein the first model training unit is further configured to:
acquiring compressed data output after the data compression processing is carried out on the model in the data end;
and returning a corresponding optimization direction to the model in the data end based on the compressed data and the first training label corresponding to the data end, so that the model in the data end is trained according to the optimization direction.
13. The apparatus of claim 12, wherein the first model training unit is further configured to:
and in response to the fact that the model in the data end is determined to be not trained well, returning a corresponding optimization direction to the model in the data end based on the trained models of other data ends in each data end, the compressed data and the first training label corresponding to the data end.
14. The apparatus of claim 12, wherein the first model training unit is further configured to:
and in response to determining that the data end is not disconnected and has no delay, returning a corresponding optimization direction to the model in the data end based on the compressed data and the first training label corresponding to the data end.
15. The apparatus of claim 11, wherein the data acquisition unit is further configured to:
determining a target data end which is not disconnected and has no delay in each data end;
and acquiring training data sent by the target data terminal according to the trained model.
16. The apparatus of claim 12, wherein the first model training unit is further configured to:
determining at least one same group of data terminals corresponding to the data terminals;
acquiring the compressed data of the same group sent by each data end of the same group;
integrating the same group of data and the compressed data by using a coordinator corresponding to the data end to obtain integrated data;
and returning a corresponding optimization direction to the model in the data end based on the integrated data and the first training label corresponding to the data end.
17. The apparatus of any of claims 11 to 16, wherein the second model training unit is further configured to:
selecting sample data from the training data, and executing the following training steps: inputting the sample data into an initial model to obtain an output result output by the initial model; determining a loss value of a loss function of the initial model based on the output result and the second training label; and in response to the loss value meeting a preset convergence condition, taking the initial model as a trained integrated model.
18. The apparatus of claim 17, wherein the second model training unit is further configured to:
and responding to the loss value not meeting the preset convergence condition, adjusting related parameters in the initial model, reselecting sample data in the training data, and continuing to execute the training step.
19. A model training device based on federal learning is applied to a data end, and the device comprises:
the data processing unit is configured to input original data into a model to be trained to obtain compressed data which is output by the model to be trained and is subjected to compression processing;
the data sending unit is configured to send the compressed data to a service end so that the service end returns an optimization direction according to the compressed data;
the third model training unit is configured to train the model to be trained according to the optimization direction to obtain a trained model;
a data acquisition unit configured to acquire training data output by the trained model;
the data sending unit is further configured to send the training data to the business end, so that the business end trains an integration model based on the training data.
20. The apparatus of claim 19, wherein the compressed data is one-dimensional data.
21. A federal learning based model training device comprising:
one or more processors;
a storage device having one or more programs stored thereon,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-10.
22. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-10.
CN202110233577.7A 2021-03-03 Model training method, device, equipment and medium based on federal learning Active CN113792883B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110233577.7A CN113792883B (en) 2021-03-03 Model training method, device, equipment and medium based on federal learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110233577.7A CN113792883B (en) 2021-03-03 Model training method, device, equipment and medium based on federal learning

Publications (2)

Publication Number Publication Date
CN113792883A true CN113792883A (en) 2021-12-14
CN113792883B CN113792883B (en) 2024-04-16

Family

ID=

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116796860A (en) * 2023-08-24 2023-09-22 腾讯科技（深圳）有限公司 Federal learning method, federal learning device, electronic equipment and storage medium

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110633806A (en) * 2019-10-21 2019-12-31 深圳前海微众银行股份有限公司 Longitudinal federated learning system optimization method, device, equipment and readable storage medium
CN111160572A (en) * 2020-04-01 2020-05-15 支付宝(杭州)信息技术有限公司 Multi-label-based federal learning method, device and system
CN111275207A (en) * 2020-02-10 2020-06-12 深圳前海微众银行股份有限公司 Semi-supervision-based horizontal federal learning optimization method, equipment and storage medium
US20200218937A1 (en) * 2019-01-03 2020-07-09 International Business Machines Corporation Generative adversarial network employed for decentralized and confidential ai training
CN111461345A (en) * 2020-03-31 2020-07-28 北京百度网讯科技有限公司 Deep learning model training method and device
CN111582505A (en) * 2020-05-14 2020-08-25 深圳前海微众银行股份有限公司 Federal modeling method, device, equipment and computer readable storage medium
US20200272899A1 (en) * 2019-02-22 2020-08-27 Ubotica Technologies Limited Systems and Methods for Deploying and Updating Neural Networks at the Edge of a Network
CN112085159A (en) * 2020-07-24 2020-12-15 西安电子科技大学 User tag data prediction system, method and device and electronic equipment
WO2021022707A1 (en) * 2019-08-06 2021-02-11 深圳前海微众银行股份有限公司 Hybrid federated learning method and architecture

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200218937A1 (en) * 2019-01-03 2020-07-09 International Business Machines Corporation Generative adversarial network employed for decentralized and confidential ai training
US20200272899A1 (en) * 2019-02-22 2020-08-27 Ubotica Technologies Limited Systems and Methods for Deploying and Updating Neural Networks at the Edge of a Network
WO2021022707A1 (en) * 2019-08-06 2021-02-11 深圳前海微众银行股份有限公司 Hybrid federated learning method and architecture
CN110633806A (en) * 2019-10-21 2019-12-31 深圳前海微众银行股份有限公司 Longitudinal federated learning system optimization method, device, equipment and readable storage medium
CN111275207A (en) * 2020-02-10 2020-06-12 深圳前海微众银行股份有限公司 Semi-supervision-based horizontal federal learning optimization method, equipment and storage medium
CN111461345A (en) * 2020-03-31 2020-07-28 北京百度网讯科技有限公司 Deep learning model training method and device
CN111160572A (en) * 2020-04-01 2020-05-15 支付宝(杭州)信息技术有限公司 Multi-label-based federal learning method, device and system
CN111582505A (en) * 2020-05-14 2020-08-25 深圳前海微众银行股份有限公司 Federal modeling method, device, equipment and computer readable storage medium
CN112085159A (en) * 2020-07-24 2020-12-15 西安电子科技大学 User tag data prediction system, method and device and electronic equipment

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
芦效峰等: "一种面向边缘计算的高效异步联邦学习机制", 计算机研究与发展, vol. 57, no. 12 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116796860A (en) * 2023-08-24 2023-09-22 腾讯科技（深圳）有限公司 Federal learning method, federal learning device, electronic equipment and storage medium
CN116796860B (en) * 2023-08-24 2023-12-12 腾讯科技（深圳）有限公司 Federal learning method, federal learning device, electronic equipment and storage medium

Similar Documents

Publication Publication Date Title
US10762443B2 (en) Crowdsourcing system with community learning
WO2020094060A1 (en) Recommendation method and apparatus
CN111144577A (en) Method and device for generating node representation in heterogeneous graph and electronic equipment
CN111539514A (en) Method and apparatus for generating structure of neural network
CN111582454B (en) Method and device for generating neural network model
CN111259222B (en) Article recommendation method, system, electronic equipment and storage medium
CN111931591A (en) Method and device for constructing key point learning model, electronic equipment and readable storage medium
CN111695698B (en) Method, apparatus, electronic device, and readable storage medium for model distillation
CN112016632A (en) Model joint training method, device, equipment and storage medium
CN111695699B (en) Method, apparatus, electronic device, and readable storage medium for model distillation
CN111507111B (en) Pre-training method and device of semantic representation model, electronic equipment and storage medium
CN112003778B (en) Message processing method, device, equipment and computer storage medium
CN113762328A (en) Model training method, device, equipment and storage medium based on federal learning
US20230281513A1 (en) Data model training method and apparatus
CN112819497B (en) Conversion rate prediction method, conversion rate prediction device, conversion rate prediction apparatus, and storage medium
Alsamhi et al. SemCom for Metaverse: Challenges, Opportunities and Future Trends
CN113792883A (en) Model training method, device, equipment and medium based on federal learning
CN112561059A (en) Method and apparatus for model distillation
CN113792883B (en) Model training method, device, equipment and medium based on federal learning
CN112269928A (en) User recommendation method and device, electronic equipment and computer readable medium
CN112101204A (en) Training method of generative countermeasure network, image processing method, device and equipment
WO2019235161A1 (en) Data analysis system and data analysis method
CN111160552A (en) Negative sampling processing method, device, equipment and computer storage medium
CN113435516B (en) Data classification method and device
CN112598139B (en) Category encoding method, category encoding device, category encoding apparatus, category encoding device, category encoding storage medium, and category encoding program product

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant