CN113792883B - Model training method, device, equipment and medium based on federal learning - Google Patents

Model training method, device, equipment and medium based on federal learning Download PDF

Info

Publication number
CN113792883B
CN113792883B CN202110233577.7A CN202110233577A CN113792883B CN 113792883 B CN113792883 B CN 113792883B CN 202110233577 A CN202110233577 A CN 202110233577A CN 113792883 B CN113792883 B CN 113792883B
Authority
CN
China
Prior art keywords
data
training
model
trained
data end
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110233577.7A
Other languages
Chinese (zh)
Other versions
CN113792883A (en
Inventor
李怡欣
张一凡
王虎
黄志翔
彭南博
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jingdong Technology Holding Co Ltd
Original Assignee
Jingdong Technology Holding Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jingdong Technology Holding Co Ltd filed Critical Jingdong Technology Holding Co Ltd
Priority to CN202110233577.7A priority Critical patent/CN113792883B/en
Publication of CN113792883A publication Critical patent/CN113792883A/en
Application granted granted Critical
Publication of CN113792883B publication Critical patent/CN113792883B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/243—Classification techniques relating to the number of classes
    • G06F18/2431—Multiple classes

Abstract

The application discloses a model training method, device, equipment and medium based on federal learning, and a specific implementation mode of the method comprises the following steps: acquiring a first training label set and a second training label from an expert experience label library, wherein the expert experience label library comprises a plurality of training labels set based on expert experience, and the first training label set comprises a plurality of different first training labels; for each data end, determining a first training label corresponding to the data end in a first training label set, and training a model in the data end based on the first training label corresponding to the data end; acquiring training data sent by each data end according to a trained model; and training the integrated model according to the training data and the second training label so that the trained integrated model integrates the data sent by each data terminal. The embodiment can improve the model training effect.

Description

Model training method, device, equipment and medium based on federal learning
Technical Field
The application relates to the field of computer technology, in particular to the field of artificial intelligence, and especially relates to a model training method, device, equipment and medium based on federal learning.
Background
Currently, data is particularly important for machine learning. The federal learning is performed by aggregating the multiparty data, so that the model training effect in the machine learning can be improved.
The model training method based on federal learning is usually based on a single label, and if label content of the single label is set unreasonably, the model training effect is poor.
Disclosure of Invention
A model training method, apparatus, device and medium based on federal learning are provided.
In a first aspect, an embodiment of the present disclosure provides a model training method based on federal learning, applied to a service end, including: acquiring a first training label set and a second training label from an expert experience label library, wherein the expert experience label library comprises a plurality of training labels set based on expert experience, and the first training label set comprises a plurality of different first training labels; for each data end, determining a first training label corresponding to the data end in a first training label set, and training a model in the data end based on the first training label corresponding to the data end; acquiring training data sent by each data end according to a trained model; and training the integrated model according to the training data and the second training label so that the trained integrated model integrates the data sent by each data terminal.
In some embodiments, training the model in the data end based on the first training label corresponding to the data end includes: obtaining compressed data output after data compression processing is carried out on the model in the data terminal; based on the compressed data and the first training label corresponding to the data end, returning a corresponding optimization direction to the model in the data end so that the model in the data end can be trained according to the optimization direction.
In some embodiments, returning the corresponding optimization directions to the model in the data end based on the compressed data and the first training label corresponding to the data end includes: and responding to the fact that the model in the data end is not trained, and returning the corresponding optimization direction to the model in the data end based on the model trained by other data ends in each data end, the compressed data and the first training label corresponding to the data end.
In some embodiments, returning the corresponding optimization directions to the model in the data end based on the compressed data and the first training label corresponding to the data end includes: and responding to the fact that the data end is not disconnected and delay does not occur, and returning a corresponding optimization direction to a model in the data end based on the compressed data and the first training label corresponding to the data end.
In some embodiments, obtaining training data sent by each data end according to a trained model includes: determining target data ends which are not dropped and delayed in each data end; and acquiring training data sent by the target data end according to the trained model.
In some embodiments, returning the corresponding optimization directions to the model in the data end based on the compressed data and the first training label corresponding to the data end includes: determining at least one data end in the same group corresponding to the data end; acquiring compressed same-group data sent by each same-group data terminal; integrating the same group of data and compressed data by utilizing a coordinator corresponding to the data end to obtain integrated data; and returning a corresponding optimization direction to the model in the data end based on the integrated data and the first training label corresponding to the data end.
In some embodiments, training the integral mold pattern based on the training data and the second training tag comprises: sample data is selected from the training data, and the following training steps are executed: inputting the sample data into an initial model to obtain an output result output by the initial model; determining a loss value of a loss function of the initial model based on the output result and the second training label; and responding to the loss value meeting a preset convergence condition, and taking the initial model as a trained integrated model.
In some embodiments, the above method further comprises: and adjusting related parameters in the initial model in response to the loss value not meeting the preset convergence condition, and re-selecting sample data from the training data to continue to execute the training step.
In a second aspect, embodiments of the present disclosure provide a model training method based on federal learning, applied to a data end, including: inputting the original data into a model to be trained to obtain compressed data after compression processing output by the model to be trained; the compressed data is sent to the service end, so that the service end returns to the optimization direction according to the compressed data; training a model to be trained according to the optimization direction to obtain a trained model; acquiring training data output by a trained model; and sending the training data to the service end so that the service end trains the integration model based on the training data.
In some embodiments, the above method further comprises: the compressed data is one-dimensional data.
In a third aspect, an embodiment of the present disclosure provides a model training device based on federal learning, which is applied to a service end, and includes: the tag acquisition unit is configured to acquire a first training tag set and a second training tag from an expert experience tag library, wherein the expert experience tag library comprises a plurality of training tags set based on expert experiences, and the first training tag set comprises a plurality of different first training tags; the first model training unit is configured to determine a first training label corresponding to each data end in the first training label set, and train the model in the data end based on the first training label corresponding to the data end; the data acquisition unit is configured to acquire training data sent by each data end according to the trained model; and the second model training unit is configured to train the integrated model according to the training data and the second training label so that the trained integrated model integrates the data sent by each data end.
In some embodiments, the first model training unit is further configured to: obtaining compressed data output after data compression processing is carried out on the model in the data terminal; based on the compressed data and the first training label corresponding to the data end, returning a corresponding optimization direction to the model in the data end so that the model in the data end can be trained according to the optimization direction.
In some embodiments, the first model training unit is further configured to: and responding to the fact that the model in the data end is not trained, and returning the corresponding optimization direction to the model in the data end based on the model trained by other data ends in each data end, the compressed data and the first training label corresponding to the data end.
In some embodiments, the first model training unit is further configured to: and responding to the fact that the data end is not disconnected and delay does not occur, and returning a corresponding optimization direction to a model in the data end based on the compressed data and the first training label corresponding to the data end.
In some embodiments, the data acquisition unit is further configured to: determining target data ends which are not dropped and delayed in each data end; and acquiring training data sent by the target data end according to the trained model.
In some embodiments, the first model training unit is further configured to: determining at least one data end in the same group corresponding to the data end; acquiring compressed same-group data sent by each same-group data terminal; integrating the same group of data and compressed data by utilizing a coordinator corresponding to the data end to obtain integrated data; and returning a corresponding optimization direction to the model in the data end based on the integrated data and the first training label corresponding to the data end.
In some embodiments, the second model training unit is further configured to: sample data is selected from the training data, and the following training steps are executed: inputting the sample data into an initial model to obtain an output result output by the initial model; determining a loss value of a loss function of the initial model based on the output result and the second training label; and responding to the loss value meeting a preset convergence condition, and taking the initial model as a trained integrated model.
In some embodiments, the second model training unit is further configured to: and adjusting related parameters in the initial model in response to the loss value not meeting the preset convergence condition, and re-selecting sample data from the training data to continue to execute the training step.
In a fourth aspect, embodiments of the present disclosure provide a model training apparatus based on federal learning, applied to a data end, including: the data processing unit is configured to input the original data into a model to be trained to obtain compressed data after compression processing output by the model to be trained; the data transmitting unit is configured to transmit the compressed data to the service end so that the service end returns to the optimization direction according to the compressed data; the third model training unit is configured to train the model to be trained according to the optimization direction to obtain a trained model; the data acquisition unit is configured to acquire training data output by the trained model; and the data transmitting unit is further configured to transmit the training data to the service end so that the service end trains the integration model based on the training data.
In some embodiments, the compressed data is one-dimensional data.
In a fifth aspect, embodiments of the present disclosure provide a model training apparatus based on federal learning, comprising: one or more processors; a storage means for storing one or more programs; the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method as in any of the first or second aspects.
In a sixth aspect, embodiments of the present disclosure provide a computer readable storage medium having stored thereon a computer program, characterized in that the program, when executed by a processor, implements a method as in any of the first or second aspects.
According to the technology of the application, the model training method based on federal learning is provided, and model training can be performed by combining a plurality of training labels set based on expert experience in an expert experience label library. The process integrates expert experience, realizes multidimensional training of the model based on a plurality of training labels, and can improve the training effect of the model.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.
Drawings
The drawings are for better understanding of the present solution and do not constitute a limitation of the present application. Wherein:
FIG. 1 is an exemplary system architecture diagram in which an embodiment of the present application may be applied;
FIG. 2 is a flow chart of one embodiment of a federal learning-based model training method according to the present application;
FIG. 3 is a flow chart of another embodiment of a federally learning-based model training method according to the present application;
FIG. 4 is a schematic illustration of one application scenario of a federally learning-based model training method according to the present application;
FIG. 5 is a schematic structural view of one embodiment of a federal learning-based model training apparatus according to the present application;
FIG. 6 is a schematic structural view of one embodiment of a federally learning-based model training apparatus according to the present application;
FIG. 7 is a block diagram of a federal learning-based model training apparatus for implementing the federal learning-based model training method of an embodiment of the present application.
Detailed Description
Exemplary embodiments of the present application are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present application to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
It should be noted that, in the case of no conflict, the embodiments and features in the embodiments may be combined with each other. The present application will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
FIG. 1 illustrates an exemplary system architecture 100 to which embodiments of the federal learning-based model training method or federal learning-based model training apparatus of the present application may be applied.
As shown in fig. 1, a system architecture 100 may include data ends 101, 102, 103, a network 104, and a service end 105. The network 104 serves as a medium providing communication links between the data terminals 101, 102, 103 and the service terminal 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
Data of different dimensions can be stored in the data terminals 101, 102 and 103, and a label corresponding to the data can be stored in the service terminal 105. The system architecture 100 can be applied to federal learning-based model training scenarios where data of different dimensions stored in different data ends need to be aggregated, the model trained together, and in the process data privacy in each data end needs to be protected. For example, the data stored in the data terminal 101 may be a user characteristic of the user group in the a dimension, the data stored in the data terminal 102 may be a user characteristic of the user group in the B dimension, the data stored in the data terminal 103 may be a user characteristic of the user group in the C dimension, and the service terminal 105 stores a user tag corresponding to the user group. Under the scene of model training based on federal learning, the user characteristics of the user group in the dimension A, the dimension B and the dimension C can be aggregated, the model is trained, and the obtained model can output more accurate user classification information.
In this implementation manner, in the process of performing model training by using the system architecture 100, the data terminals 101, 102, 103 need to send compressed data after performing data compression processing to the service terminal 105 through the network 104, where the compressed data may be compressed data output by the models in the data terminals 101, 102, 103 when the models in the data terminals are trained randomly. The service end 105 may receive the compressed data after the data compression processing sent by the data ends 101, 102, 103, and determine, based on the compressed data and the plurality of first training labels, an optimization direction of the model in the data end 101, an optimization direction of the model in the data end 102, and an optimization direction of the model in the data end 103, and send the optimization directions to the data ends 101, 102, 103. After receiving the optimization direction, the data terminals 101, 102, 103 can train the models in the data terminals 101, 102, 103 based on the optimization direction to obtain a trained model in the data terminal 101, a trained model in the data terminal 102, and a trained model in the data terminal 103. At this point, training of the model in the data terminals 101, 102, 103 is completed.
Further, the data terminals 101, 102, 103 may send training data to the service terminal 105 based on the trained model, where the training data includes data output by the trained model in the data terminal 101, data output by the trained model in the data terminal 102, and data output by the trained model in the data terminal 103. The service end 105 may receive the training data, and train the integrated model based on the training data and a preset second training label, to obtain a trained integrated model. The first training label and the second training label may be the same label or different labels, which is not limited in this embodiment.
The data terminals 101, 102, 103 may be hardware or software. When the data terminals 101, 102, 103 are hardware, they may be implemented as a distributed data terminal cluster formed by a plurality of data terminals, or may be implemented as a single data terminal. When the data terminals 101, 102, 103 are software, they may be implemented as a plurality of software or software modules (e.g. for providing distributed services), or as a single software or software module. The present invention is not particularly limited herein.
The service end 105 may be hardware or software. When the service end 105 is hardware, it may be implemented as a distributed service end cluster formed by multiple service ends, or may be implemented as a single service end. When the business side 105 is software, it may be implemented as a plurality of software or software modules (e.g., to provide distributed services), or as a single software or software module. The present invention is not particularly limited herein.
It should be noted that, the model training method based on federal learning provided in the embodiments of the present application may be executed by the data end 101, 102, 103, or may be executed by the service end 105. Accordingly, the model training device based on federal learning can be set in the data end 101, 102, 103 or the service end 105. The present invention is not particularly limited herein.
It should be understood that the numbers of data ends, networks, and traffic ends in fig. 1 are merely illustrative. There may be any number of data, network and service ends, as desired for implementation.
With continued reference to FIG. 2, a flow 200 of one embodiment of a federally learning-based model training method applied to a business end in accordance with the present application is shown. The model training method based on federal learning of the embodiment comprises the following steps:
step 201, a first training label set and a second training label are obtained from a specialist experience label library, wherein the specialist experience label library comprises a plurality of training labels set based on specialist experience, and the first training label set comprises a plurality of different first training labels.
In this embodiment, the data end may be a device storing data, and the service end may be a device storing a data tag. An expert experience tag library is arranged in an execution main body (a business end shown in fig. 1) of the model training method based on federal learning. When model training is carried out, a required training label can be selected from the expert experience label library according to training requirements and used as a first training label set and a second training label. The first training label in the first training label set may be the same as the second training label, or may be different from the second training label, which is not limited in this embodiment.
It should be noted that, the service end may be a device corresponding to a certain data end in the at least one data end, or the service end may be another device independent of the at least one data end, which is not limited in this embodiment.
Step 202, for each data end, determining a first training label corresponding to the data end in a first training label set, and training a model in the data end based on the first training label corresponding to the data end.
In this embodiment, the execution body is able to determine a first training tag corresponding to each data end in the first training tag set, and train the model in the data end based on the data sent to the execution body by each data end and the first training tag corresponding to the data end.
In some optional implementations of this embodiment, training the model in the data end based on the first training tag corresponding to the data end includes: obtaining compressed data output after data compression processing is carried out on the model in the data terminal; based on the compressed data and the first training label corresponding to the data end, returning a corresponding optimization direction to the model in the data end so that the model in the data end can be trained according to the optimization direction.
In this implementation manner, the executing body is capable of receiving compressed data that is output after data compression processing and is sent by at least one data end (such as the data end shown in fig. 1) respectively, and returning a corresponding optimization direction to each data end according to the first training label corresponding to each data end. Wherein the different first training labels may correspond to different training directions. And different data ends can store data of different dimensions. Each data end is provided with a local model, the model can be a deep learning model such as a deep neural network, or a traditional machine learning model such as a decision tree, and the embodiment is not limited to the model. The data sent by the data terminals are data output by the model of each data terminal in at least one data terminal.
Further, the foregoing optimization direction is used to describe an error value between the integrated data and the training target, for example, the training target of the coordinator is that the value output by the model at the data end is a target value, after receiving the data sent by the model at the data end, the data and the target value may be compared to determine an error value, for example, the data is that the target value is added by three, and the error value is determined to be positive three, where the corresponding model optimization direction is that the value output by the model at the data end is oriented in the direction of subtracting three. Each of the at least one coordinator may integrate compressed data respectively sent by at least one data terminal, to obtain integrated data corresponding to each coordinator. And determining the optimization direction of the model of each data end in the at least one data end based on the first training label corresponding to each coordinator and the integrated data corresponding to each coordinator. Each of the coordinators may correspond to a first training label of a different dimension, so as to implement multi-dimensional model training for the model of each of the at least one data terminal. At this time, each data end in the at least one data end can receive the model optimization directions returned by the plurality of coordinators, and perform model training based on the plurality of model optimization directions, so that the accuracy of model training can be improved.
In some optional implementations of this embodiment, returning, based on the compressed data and the first training label corresponding to the data end, a corresponding optimization direction to the model in the data end includes: and responding to the fact that the model in the data end is not trained, and returning the corresponding optimization direction to the model in the data end based on the model trained by other data ends in each data end, the compressed data and the first training label corresponding to the data end.
In this implementation, when model training is performed on the models in at least one data end, the models of each data end in at least one data end are not necessarily completely synchronously trained, and there may be a case where some models are trained and some models are not trained yet, and for this case, training of the models in the data end may be assisted by using the models trained in other data ends in response to the model not being trained in the data end. That is, based on the trained model in the other data end, the compressed data sent by the data end and the first training label corresponding to the data end, the corresponding optimization direction is returned to the model in the data end. Or before model training is carried out on the models in at least one data end, the priority of each model for training can be set, when training is carried out, the model with high priority is trained firstly to obtain a trained model, and when the model with low priority is trained, the trained model with high priority, compressed data sent by the data end and the first training label set can be combined to determine the optimization direction of the model without training. The training process can use the trained model to assist in training the untrained model, so that the training speed of the untrained model can be improved. The training of the untrained model with the trained model can be realized based on a transfer learning technology, wherein the transfer learning is a method for deep learning by taking a pre-trained model as a starting point of a new model in a computer vision task and a natural language processing task.
In some optional implementations of this embodiment, returning, based on the compressed data and the first training label corresponding to the data end, a corresponding optimization direction to the model in the data end includes: and responding to the fact that the data end is not disconnected and delay does not occur, and returning a corresponding optimization direction to a model in the data end based on the compressed data and the first training label corresponding to the data end.
In this implementation manner, in the process of model training, compressed data sent by each data end in at least one data end is integrated in a service end, if at this time, one or more data ends have a disconnection or delay phenomenon, abnormal operation of the model in the data end can be determined, at this time, the service end self-training may not integrate the compressed data sent by the model in the data end, and the self-training does not train the model in the data end, but determines an optimization direction of the model normally operated in at least one data end based on the compressed data sent by the model in the data end normally operated and the first training label set. That is, the corresponding optimized direction is not returned to the data end with the disconnection or delay in the training of the round. The process can decouple at least one data end participating in training to a greater extent, and the robustness of model training is improved.
Optionally, after the service end determines the optimization direction of the current training of each data end, the optimization direction may be sent to the corresponding data end, so that each data end in the at least one data end trains the model based on the corresponding model optimization direction. And repeating the step 202 until the models in the data ends are converged, and obtaining the trained models in each data end in at least one data end. This process completes the training of the local model in the at least one data side, and the integrated model in the business side can be subsequently trained based on the trained model in the at least one data side.
In some optional implementations of this embodiment, returning, based on the compressed data and the first training label corresponding to the data end, a corresponding optimization direction to the model in the data end includes: determining at least one data end in the same group corresponding to the data end; acquiring compressed same-group data sent by each same-group data terminal; integrating the same group of data and compressed data by utilizing a coordinator corresponding to the data end to obtain integrated data; and returning a corresponding optimization direction to the model in the data end based on the integrated data and the first training label corresponding to the data end.
In this implementation manner, the service end may include at least one coordinator, where the coordinator has a corresponding first training tag. Each of the at least one coordinator is used for integrating compressed data sent by each of the at least one data terminal. Each coordinator has a corresponding first training label, and the number of the first training labels corresponding to each coordinator may be one or greater than one. The first training labels in different coordinators may correspond to training targets of different dimensions, for example, the training targets of different coordinators may be classified for different attributes, where each training target of a coordinators is to classify for the attribute corresponding to the coordinators, and the attributes for which different coordinators are different. For example, assume that there are 3 data terminals and that the service terminal has 3 cooperators, at this time, each cooperator can integrate compressed data sent by the 3 data terminals. And each coordinator also has a corresponding first training label, and at this time, the models in the data end can be trained based on different training labels by utilizing the compressed data sent by the data end of each coordinator integration. Because the first training labels corresponding to different coordinators can be training labels with different dimensionalities, the coordinators are adopted to integrate the compressed data sent by the data end, and the multidimensional training of the model in the data end can be realized. Further, each coordinator may be responsible for integrating the compressed data sent by one set of data terminals, and the at least one peer data terminal and the data terminal may be the same coordinator responsible for integrating the data terminals.
In some optional implementations of this embodiment, the manner in which each data end generates compressed data after data compression based on the local model may specifically be: the data end trains the model randomly, and random data is input to the model, so that the model outputs compressed data. The compressed data may be data obtained after data compression of random data. The data is compressed by using the model of the data end, so that the information entropy of the compressed data can be improved, and the privacy of data transmission is ensured. In addition, the encryption and decryption algorithm is not needed in the process, so that the complexity of maintaining the data security is reduced.
Step 203, obtaining training data sent by each data end according to the trained model.
In this embodiment, after the models in the data end meet the convergence condition and obtain the trained models, each data end may respectively send training data output by the trained models to the service end, and the service end may receive the training data sent by at least one data end based on the trained models.
In some optional implementations of this embodiment, acquiring training data sent by each data end according to the trained model includes: determining target data ends which are not dropped and delayed in each data end; and acquiring training data sent by the target data end according to the trained model.
In this implementation manner, after the model in the data end meets the convergence condition and the trained model is obtained, if it is determined that a certain data end has a disconnection or delay, the data end is not used when the whole die is trained, so that only training data sent by a target data end which has not been disconnected and has not been delayed need to be obtained.
And 204, training the integral model according to the training data and the second training label, so that the trained integral model integrates the data sent by each data end.
In this embodiment, the service end may train the integrated model based on the training data and the second training label.
In some optional implementations of this embodiment, training the overall mold pattern according to the training data and the second training tag includes: sample data is selected from the training data, and the following training steps are executed: inputting the sample data into an initial model to obtain an output result output by the initial model; determining a loss value of a loss function of the initial model based on the output result and the second training label; and responding to the loss value meeting a preset convergence condition, and taking the initial model as a trained integrated model.
The initial model is an initial integrated model, sample data can be selected from training data to serve as input data of the initial model, an output result output by the initial model is obtained, and a loss value of a loss function of the initial model is determined based on the output result and a second training label, wherein a loss value calculation formula is as follows:
L(ρ i ，f i ，Y)＝arg min((Y-F(ρ i ，f i )))
wherein L (ρ) i ，f i Y) refers to the loss value of the loss function ρ i Refer to super parameters, fi refers to training data, and Y refers to a second training label. Wherein the super parameter ρ i For adjusting the importance of each fi. F means for integrating each F i The integration method of (a) may specifically be summation, and the embodiment is not limited thereto. And at each f i With corresponding hyper-parameters ρ i In the case of F, F may be the same as F i And corresponding super parameter ρ i And (5) a calculation mode of weighted summation is carried out. And F, obtaining a predicted value of the integrated model for the training round, and subtracting the predicted value from Y to obtain a loss value of the training round loss function.
The loss value of the loss function of the initial model can be calculated by substituting the training data, the super parameters and the training labels of each training cycle into the loss value calculation formula. And repeating the training for multiple rounds until the loss value meets the preset convergence condition, and determining an initial model with the loss value meeting the preset convergence condition as an integrated model.
In some alternative implementations of the present embodiment, the following steps may also be performed: and adjusting related parameters in the initial model in response to the loss value not meeting a preset convergence condition, and reselecting sample data from the training data to continue to execute the training step.
If the loss value does not meet the preset convergence condition, the related parameters in the initial model can be adjusted, sample data are selected again from the training data, multiple rounds of training are started, and the training step is continuously executed.
The model training method based on federal learning provided in the above embodiment of the present application may combine a plurality of training tags set based on expert experience in an expert experience tag library to perform model training. The process integrates expert experience, realizes multidimensional training of the model based on a plurality of training labels, and can improve the training effect of the model. In addition, when the models in the data end are trained, each round of training only needs the service end to return to the model optimization direction, namely only one time of information interaction is needed, and when the models are integrated in the service end are trained, each round of training only needs to acquire data output by the trained models in the data end, and the model training can be completed through one time of information interaction. By adopting the model training method, the communication times between the service end and the data end can be reduced, so that the model training complexity is reduced.
With continued reference to FIG. 3, a flow 300 of one embodiment of a federally learning-based model training method applied to a data-side according to the present application is shown. The model training method based on federal learning comprises the following steps:
step 301, inputting the original data into the model to be trained, and obtaining compressed data after compression processing output by the model to be trained.
In this embodiment, the original data may be input data input to the model when the model is trained randomly. The original data can be high-dimensional data, and after the original data is input into the model, the model can perform data compression on the original data to obtain compressed data after compression processing output by the model. The compressed data may be one-dimensional data obtained by performing data compression on high-dimensional data. The one-dimensional compressed data is sent to the service end, so that the service end is difficult to deduce the information of the original data from the compressed data, and the data security is improved. Alternatively, the data compression may be performed based on a deep learning model or a conventional machine learning model, which is not limited in this embodiment. Wherein the compressed data may be one-dimensional data.
And step 302, the compressed data is sent to the service end, so that the service end returns to the optimization direction according to the compressed data.
In this embodiment, since the number of data ends may be greater than one, in the case that there is a data end disconnection or a data end delay that causes the data end to train the present round of training without transmitting compressed data to the service end, in the process of training the present round, the service end may ignore the data end, integrate the compressed data transmitted by other data ends, and return an optimization direction to the model in other data ends.
And step 303, training the model to be trained according to the optimization direction to obtain a trained model.
In this embodiment, the data end may adjust parameters of the model in the data end based on the optimization direction returned by the service end, and output compressed data by using the model after the parameters are adjusted until the model in the data end converges, so as to obtain a trained model.
Step 304, obtaining training data output by the trained model.
In this embodiment, after the training of the model in the service end is completed, data may be input to the trained model, the data may be pre-stored data, and after the data is input to the trained model, training data output by the trained model may be obtained. The training data may be data after data compression.
Step 305, send the training data to the service end, so that the service end trains the integration model based on the training data.
In this embodiment, after the local model is trained, the data end may input data to the trained local model to obtain training data output by the trained model, where the training data may be used as input data for the service end to train the integrated model, so that the service end realizes training of the integrated model based on the training data.
With continued reference to fig. 4, fig. 4 is a schematic diagram of an application scenario of federally learning-based model training according to the present embodiment. In the application scenario of fig. 4, there are respective models 401 in the a-group data side, respective models 402 in the B-group data side, and respective models 403 in the C-group data side. Each model 401 in the a-group data side may send compressed data to a coordinator D404 in the service side, each model 402 in the B-group data side may send compressed data to a coordinator E405 in the service side, and each model 403 in the C-group data side may send compressed data to a coordinator F406 in the service side. It should be noted that, each model 401 in the a-group data terminal may be a model in each of the plurality of data terminals. Similarly, each model 402 in the B-group data terminal and each model 403 in the C-group data terminal may be a model in each of the plurality of data terminals. The collaborative device D404 may receive the compressed data sent by each model 401 in the data end of the group A, determine an optimization direction of each model 401 in the data end of the group A based on a first training label corresponding to the collaborative device D404, and return the optimization direction to the model in the corresponding data end, so that the model in the data end trains based on the optimization direction, and finally obtains a trained model. Similarly, the cooperator E405 and the cooperator F406 may also return corresponding optimization directions to each model 402 in the data end of the group B and each model 403 in the data end of the group C based on the received compressed data, so that the models in the data end train based on the optimization directions, and finally a trained model is obtained. Each coordinator corresponds to a corresponding first training label, and the first training labels of different coordinators can be labels with different dimensions. Meanwhile, the collaborative device D404, the collaborative device E405 and the collaborative device F406 can train the collaborative device model itself based on the data integration result of the round of training and the first training label in the process of integrating each model in the corresponding group of data ends to determine the optimization direction of each model, and finally the trained collaborative device is obtained.
After each model in the A, B, C data set is trained, training data can be obtained based on the trained model, and the training data is sent to the coordinator D404, the coordinator E405 and the coordinator F406 respectively, so that the coordinator D404, the coordinator E405 and the coordinator F406 use the training data as input data of the integrated model 407, and train the integrated model 407 based on the input data of the integrated model 407 and a preset second training label, so as to obtain a trained integrated model. In this process, the first training label and the second training label which are fused with the expert experience may be preset, and the first training label and the second training label may be the same or different. The expert experience tag library comprises a plurality of preset training tags for fusing expert experiences, the training tags for fusing the expert experiences can be selected based on the first training tag set and the second training tag obtained from the expert experience tag library, and the training tags are used for model training, so that the effect is better. Based on the first training label set and the second training label, the model and the integrated model in the data end are respectively trained, different training labels can be based, and multi-dimensional model training is realized. In addition, each model in the A, B, C group data end can compress high-dimensional data into one-dimensional data to be transmitted to the service end, so that data safety is guaranteed, encryption transmission is not needed to be realized based on cryptography, and model training complexity is reduced. Compared with frequent communication caused by encryption transmission based on cryptography, the method has the advantages that data compression is adopted to replace encryption processing, so that frequent communication caused by encryption and decryption processes can be reduced, the communication times between a data end and a service end are reduced, the coupling degree of each party participating in model training is reduced, and each party participating in model training can be decoupled to a certain degree.
With further reference to fig. 5, as an implementation of the method shown in the foregoing figures, the present application provides an embodiment of a model training apparatus based on federal learning, where the embodiment of the apparatus corresponds to the embodiment of the method shown in fig. 2, and the apparatus may be specifically applied to various electronic devices.
As shown in fig. 5, the model training apparatus 500 based on federal learning of the present embodiment includes: a label acquisition unit 501, a first model training unit 502, a data acquisition unit 503, and a second model training unit 504.
A tag obtaining unit 501 configured to obtain a first training tag set and a second training tag in an expert experience tag library, where the expert experience tag library includes a plurality of training tags set based on expert experiences, and the first training tag set includes a plurality of different first training tags; the first model training unit 502 is configured to determine, for each data end, a first training label corresponding to the data end in the first training label set, and train a model in the data end based on the first training label corresponding to the data end; a data obtaining unit 503, configured to obtain training data sent by each data end according to the trained model; and the second model training unit 504 is configured to train the integrated model according to the training data and the second training label, so that the trained integrated model integrates the data sent by each data end.
In some optional implementations of the present embodiment, the first model training unit 502 is further configured to: obtaining compressed data output after data compression processing is carried out on the model in the data terminal; based on the compressed data and the first training label corresponding to the data end, returning a corresponding optimization direction to the model in the data end so that the model in the data end can be trained according to the optimization direction.
In some optional implementations of the present embodiment, the first model training unit 502 is further configured to: and responding to the fact that the model in the data end is not trained, and returning the corresponding optimization direction to the model in the data end based on the model trained by other data ends in each data end, the compressed data and the first training label corresponding to the data end.
In some optional implementations of the present embodiment, the first model training unit 502 is further configured to: and responding to the fact that the data end is not disconnected and delay does not occur, and returning a corresponding optimization direction to a model in the data end based on the compressed data and the first training label corresponding to the data end.
In some optional implementations of the present embodiment, the data acquisition unit is further configured to: determining target data ends which are not dropped and delayed in each data end; and acquiring training data sent by the target data end according to the trained model.
In some optional implementations of the present embodiment, the first model training unit 502 is further configured to: determining at least one data end in the same group corresponding to the data end; acquiring compressed same-group data sent by each same-group data terminal; integrating the same group of data and compressed data by utilizing a coordinator corresponding to the data end to obtain integrated data; and returning a corresponding optimization direction to the model in the data end based on the integrated data and the first training label corresponding to the data end.
In some optional implementations of the present embodiment, the second model training unit 504 is further configured to: sample data is selected from the training data, and the following training steps are executed: inputting the sample data into an initial model to obtain an output result output by the initial model; determining a loss value of a loss function of the initial model based on the output result and the second training label; and responding to the loss value meeting a preset convergence condition, and taking the initial model as a trained integrated model.
In some optional implementations of the present embodiment, the second model training unit 504 is further configured to: and adjusting related parameters in the initial model in response to the loss value not meeting the preset convergence condition, and re-selecting sample data from the training data to continue to execute the training step.
It should be appreciated that the elements 501 to 504 recited in the federal learning based model training arrangement 500 correspond to the respective steps in the method described with reference to fig. 2. Thus, the operations and features described above for the federal learning-based model training method are equally applicable to the apparatus 500 and the elements contained therein, and are not described in detail herein.
With further reference to fig. 6, as an implementation of the method shown in the foregoing figures, the present application provides an embodiment of a model training apparatus based on federal learning, where the embodiment of the apparatus corresponds to the embodiment of the method shown in fig. 3, and the apparatus may be specifically applied to various electronic devices.
As shown in fig. 6, the model training apparatus 600 based on federal learning of the present embodiment includes: a data processing unit 601, a data transmitting unit 602, a third model training unit 603, a data acquiring unit 604, and a data transmitting unit 605.
The data processing unit 601 is configured to input the original data into a model to be trained, and obtain compressed data after compression processing output by the model to be trained; a data transmitting unit 602 configured to transmit the compressed data to the service end, so that the service end returns an optimized direction according to the compressed data; a third model training unit 603 configured to train the model to be trained according to the optimization direction, resulting in a trained model; a data acquisition unit 604 configured to acquire training data output by the trained model; the data transmitting unit 605 is further configured to transmit the training data to the service end, so that the service end trains the integration model based on the training data.
In some alternative implementations of the present embodiment, the compressed data is one-dimensional data.
According to embodiments of the present application, there is also provided a model training apparatus based on federal learning and a readable storage medium.
Referring now to FIG. 7, a block diagram of a federally learning-based model training appliance (e.g., business or data side of FIG. 1) 700 is shown, in accordance with an embodiment of the present application. The data side/service side shown in fig. 7 is only one example, and should not impose any limitation on the functionality and scope of use of the embodiments of the present disclosure. Model training devices based on federal learning are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the application described and/or claimed herein.
As shown in fig. 7, the model training apparatus based on federal learning includes: one or more processors 701, memory 702, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions executing within the federal learning-based model training apparatus, including instructions stored in or on memory to display graphical information of the GUI on an external input/output device, such as a display device coupled to the interface. In other embodiments, multiple processors and/or multiple buses may be used, if desired, along with multiple memories and multiple memories. Also, multiple federal learning-based model training devices can be connected, each providing a portion of the necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system). One processor 701 is illustrated in fig. 7.
Memory 702 is a non-transitory computer-readable storage medium provided herein. The memory stores instructions executable by the at least one processor to cause the at least one processor to perform the federal learning-based model training method provided herein. The non-transitory computer readable storage medium of the present application stores computer instructions for causing a computer to perform the federal learning-based model training method provided herein.
The memory 702 is used as a non-transitory computer readable storage medium, and may be used to store a non-transitory software program, a non-transitory computer executable program, and a module, such as program instructions/modules corresponding to the federal learning-based model training method in the embodiments of the present application (e.g., the tag acquisition unit 501, the first model training unit 502, the data acquisition unit 503, and the second model training unit 504 shown in fig. 5, or the data processing unit 601, the data transmission unit 602, the third model training unit 603, the data acquisition unit 604, and the data transmission unit 605 shown in fig. 6). The processor 701 executes various functional applications and data processing of the federal learning-based model training apparatus by running non-transitory software programs, instructions, and modules stored in the memory 702, i.e., implements the federal learning-based model training method in the above-described method embodiments.
Memory 702 may include a storage program area that may store an operating system, at least one application program required for functionality, and a storage data area; the storage data area may store data created from the use of federal learning-based model training devices, and the like. In addition, the memory 702 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 702 may optionally include memory located remotely from processor 701, which may be connected to the electronic device executing the process for storing data via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The model training apparatus based on federal learning may further include: an input device 703 and an output device 704. The processor 701, the memory 702, the input device 703 and the output device 704 may be connected by a bus or otherwise, in fig. 7 by way of example.
The input device 703 may receive input numeric or character information and generate key signal inputs related to performing user settings and function controls of the electronic device for storing data, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, a pointer stick, one or more mouse buttons, a track ball, a joystick, and the like. The output device 704 may include a display apparatus, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors), among others. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device may be a touch screen.
Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific ASIC (application specific integrated circuit), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
These computing programs (also referred to as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented in a high-level procedural and/or object-oriented programming language, and/or in assembly/machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
According to the technical scheme of the embodiment of the application, the model training method based on federal learning is provided, and the model training effect can be improved.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present application may be performed in parallel, sequentially, or in a different order, provided that the desired results of the technical solutions disclosed in the present application can be achieved, and are not limited herein.
The above embodiments do not limit the scope of the application. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application are intended to be included within the scope of the present application.

Claims (20)

1. A model training method based on federal learning, applied to a service end, the method comprising:
acquiring a first training label set and a second training label from an expert experience label library, wherein the expert experience label library comprises a plurality of training labels set based on expert experience, and the first training label set comprises a plurality of different first training labels;
for each data end, determining a first training label corresponding to the data end in the first training label set, and training a model in the data end based on the first training label corresponding to the data end;
acquiring training data sent by each data end according to a trained model;
training the integral die type according to the training data and the second training label so that the trained integral die type integrates the data sent by each data end;
the training of the model in the data end based on the first training label corresponding to the data end comprises the following steps:
obtaining compressed data output after data compression processing is carried out on the model in the data terminal;
and returning a corresponding optimization direction to the model in the data end based on the compressed data and the first training label corresponding to the data end so as to train the model in the data end according to the optimization direction, wherein the optimization direction is used for describing an error value between the integrated data and a training target.
2. The method of claim 1, wherein the returning the corresponding optimized direction to the model in the data end based on the compressed data and the first training label corresponding to the data end comprises:
and responding to the fact that the model in the data end is not trained, and returning a corresponding optimization direction to the model in the data end based on the models trained by other data ends in each data end, the compressed data and the first training labels corresponding to the data end.
3. The method of claim 1, wherein the returning the corresponding optimized direction to the model in the data end based on the compressed data and the first training label corresponding to the data end comprises:
and responding to the fact that the data end is not disconnected and delay does not occur, and returning a corresponding optimization direction to a model in the data end based on the compressed data and the first training label corresponding to the data end.
4. The method of claim 1, wherein the obtaining training data sent by each data end according to the trained model comprises:
determining target data ends which are not dropped and delayed in each data end;
and acquiring training data sent by the target data end according to the trained model.
5. The method of claim 1, wherein the returning the corresponding optimized direction to the model in the data end based on the compressed data and the first training label corresponding to the data end comprises:
determining at least one data end in the same group corresponding to the data end;
acquiring compressed same-group data sent by each same-group data terminal;
integrating the same group of data and the compressed data by utilizing a coordinator corresponding to the data end to obtain integrated data;
and returning a corresponding optimization direction to the model in the data end based on the integrated data and the first training label corresponding to the data end.
6. The method of any of claims 1 to 5, wherein the training the integral pattern according to the training data and the second training tag comprises:
sample data is selected from the training data, and the following training steps are executed: inputting the sample data into an initial model to obtain an output result output by the initial model; determining a loss value of a loss function of the initial model based on the output result and the second training label; and responding to the loss value meeting a preset convergence condition, and taking the initial model as a trained integrated model.
7. The method of claim 6, wherein the method further comprises:
and in response to the loss value not meeting the preset convergence condition, adjusting related parameters in the initial model, and re-selecting sample data from the training data to continue to execute the training step.
8. A model training method based on federal learning, applied to a data end, the method comprising:
inputting the original data into a model to be trained to obtain compressed data after compression processing output by the model to be trained;
the compressed data is sent to a service end, so that the service end returns an optimization direction according to the compressed data, wherein the optimization direction is used for describing an error value between the integrated data and a training target;
training the model to be trained according to the optimization direction to obtain a trained model;
acquiring training data output by the trained model;
and sending the training data to the service end so that the service end trains an integration model based on the training data.
9. The method of claim 8, wherein the compressed data is one-dimensional data.
10. A model training device based on federal learning, applied to a business end, the device comprising:
A tag acquisition unit configured to acquire a first training tag set and a second training tag in an expert experience tag library, the expert experience tag library including a plurality of training tags set based on expert experiences, the first training tag set including a plurality of different first training tags;
the first model training unit is configured to determine a first training label corresponding to each data end in the first training label set, and train a model in the data end based on the first training label corresponding to the data end;
the data acquisition unit is configured to acquire training data sent by each data end according to the trained model;
the second model training unit is configured to train the integral model according to the training data and the second training label so that the trained integral model integrates the data sent by each data end;
wherein the first model training unit is further configured to:
obtaining compressed data output after data compression processing is carried out on the model in the data terminal;
and returning a corresponding optimization direction to the model in the data end based on the compressed data and the first training label corresponding to the data end so as to train the model in the data end according to the optimization direction, wherein the optimization direction is used for describing an error value between the integrated data and a training target.
11. The apparatus of claim 10, wherein the first model training unit is further configured to:
and responding to the fact that the model in the data end is not trained, and returning a corresponding optimization direction to the model in the data end based on the models trained by other data ends in each data end, the compressed data and the first training labels corresponding to the data end.
12. The apparatus of claim 10, wherein the first model training unit is further configured to:
and responding to the fact that the data end is not disconnected and delay does not occur, and returning a corresponding optimization direction to a model in the data end based on the compressed data and the first training label corresponding to the data end.
13. The apparatus of claim 10, wherein the data acquisition unit is further configured to:
determining target data ends which are not dropped and delayed in each data end;
and acquiring training data sent by the target data end according to the trained model.
14. The apparatus of claim 10, wherein the first model training unit is further configured to:
determining at least one data end in the same group corresponding to the data end;
Acquiring compressed same-group data sent by each same-group data terminal;
integrating the same group of data and the compressed data by utilizing a coordinator corresponding to the data end to obtain integrated data;
and returning a corresponding optimization direction to the model in the data end based on the integrated data and the first training label corresponding to the data end.
15. The apparatus of any of claims 10 to 14, wherein the second model training unit is further configured to:
sample data is selected from the training data, and the following training steps are executed: inputting the sample data into an initial model to obtain an output result output by the initial model; determining a loss value of a loss function of the initial model based on the output result and the second training label; and responding to the loss value meeting a preset convergence condition, and taking the initial model as a trained integrated model.
16. The apparatus of claim 15, wherein the second model training unit is further configured to:
and in response to the loss value not meeting the preset convergence condition, adjusting related parameters in the initial model, and re-selecting sample data from the training data to continue to execute the training step.
17. A model training device based on federal learning, applied to a data end, the device comprising:
the data processing unit is configured to input the original data into a model to be trained to obtain compressed data after compression processing output by the model to be trained;
the data sending unit is configured to send the compressed data to a service end so that the service end returns an optimization direction according to the compressed data, wherein the optimization direction is used for describing an error value between the integrated data and a training target;
the third model training unit is configured to train the model to be trained according to the optimization direction to obtain a trained model;
the data acquisition unit is configured to acquire training data output by the trained model;
the data transmitting unit is further configured to transmit the training data to the service end, so that the service end trains an integration model based on the training data.
18. The apparatus of claim 17, wherein the compressed data is one-dimensional data.
19. A model training apparatus based on federal learning, comprising:
one or more processors;
a storage device having one or more programs stored thereon,
The one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1-9.
20. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-9.
CN202110233577.7A 2021-03-03 2021-03-03 Model training method, device, equipment and medium based on federal learning Active CN113792883B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110233577.7A CN113792883B (en) 2021-03-03 2021-03-03 Model training method, device, equipment and medium based on federal learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110233577.7A CN113792883B (en) 2021-03-03 2021-03-03 Model training method, device, equipment and medium based on federal learning

Publications (2)

Publication Number Publication Date
CN113792883A CN113792883A (en) 2021-12-14
CN113792883B true CN113792883B (en) 2024-04-16

Family

ID=78876881

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110233577.7A Active CN113792883B (en) 2021-03-03 2021-03-03 Model training method, device, equipment and medium based on federal learning

Country Status (1)

Country Link
CN (1) CN113792883B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116796860B (en) * 2023-08-24 2023-12-12 腾讯科技（深圳）有限公司 Federal learning method, federal learning device, electronic equipment and storage medium

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110633806A (en) * 2019-10-21 2019-12-31 深圳前海微众银行股份有限公司 Longitudinal federated learning system optimization method, device, equipment and readable storage medium
CN111160572A (en) * 2020-04-01 2020-05-15 支付宝(杭州)信息技术有限公司 Multi-label-based federal learning method, device and system
CN111275207A (en) * 2020-02-10 2020-06-12 深圳前海微众银行股份有限公司 Semi-supervision-based horizontal federal learning optimization method, equipment and storage medium
CN111461345A (en) * 2020-03-31 2020-07-28 北京百度网讯科技有限公司 Deep learning model training method and device
CN111582505A (en) * 2020-05-14 2020-08-25 深圳前海微众银行股份有限公司 Federal modeling method, device, equipment and computer readable storage medium
CN112085159A (en) * 2020-07-24 2020-12-15 西安电子科技大学 User tag data prediction system, method and device and electronic equipment
WO2021022707A1 (en) * 2019-08-06 2021-02-11 深圳前海微众银行股份有限公司 Hybrid federated learning method and architecture

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11010637B2 (en) * 2019-01-03 2021-05-18 International Business Machines Corporation Generative adversarial network employed for decentralized and confidential AI training
US20200272899A1 (en) * 2019-02-22 2020-08-27 Ubotica Technologies Limited Systems and Methods for Deploying and Updating Neural Networks at the Edge of a Network

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021022707A1 (en) * 2019-08-06 2021-02-11 深圳前海微众银行股份有限公司 Hybrid federated learning method and architecture
CN110633806A (en) * 2019-10-21 2019-12-31 深圳前海微众银行股份有限公司 Longitudinal federated learning system optimization method, device, equipment and readable storage medium
CN111275207A (en) * 2020-02-10 2020-06-12 深圳前海微众银行股份有限公司 Semi-supervision-based horizontal federal learning optimization method, equipment and storage medium
CN111461345A (en) * 2020-03-31 2020-07-28 北京百度网讯科技有限公司 Deep learning model training method and device
CN111160572A (en) * 2020-04-01 2020-05-15 支付宝(杭州)信息技术有限公司 Multi-label-based federal learning method, device and system
CN111582505A (en) * 2020-05-14 2020-08-25 深圳前海微众银行股份有限公司 Federal modeling method, device, equipment and computer readable storage medium
CN112085159A (en) * 2020-07-24 2020-12-15 西安电子科技大学 User tag data prediction system, method and device and electronic equipment

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
一种面向边缘计算的高效异步联邦学习机制;芦效峰等;计算机研究与发展;第57卷(第12期);全文 *

Also Published As

Publication number Publication date
CN113792883A (en) 2021-12-14

Similar Documents

Publication Publication Date Title
CN111221984B (en) Multi-mode content processing method, device, equipment and storage medium
CN111144577B (en) Method and device for generating node representation in heterogeneous graph and electronic equipment
JP7135143B2 (en) Methods, apparatus, electronic devices and computer readable storage media for building keypoint learning models
CN111783870B (en) Human body attribute identification method, device, equipment and storage medium
CN111695699B (en) Method, apparatus, electronic device, and readable storage medium for model distillation
CN112084366B (en) Method, apparatus, device and storage medium for retrieving image
CN111695698B (en) Method, apparatus, electronic device, and readable storage medium for model distillation
CN112016632A (en) Model joint training method, device, equipment and storage medium
US20230281513A1 (en) Data model training method and apparatus
CN113762328A (en) Model training method, device, equipment and storage medium based on federal learning
CN113792883B (en) Model training method, device, equipment and medium based on federal learning
CN112382294B (en) Speech recognition method, device, electronic equipment and storage medium
CN114091572A (en) Model training method and device, data processing system and server
CN112016523B (en) Cross-modal face recognition method, device, equipment and storage medium
Alsamhi et al. SemCom for Metaverse: Challenges, Opportunities and Future Trends
CN112819497B (en) Conversion rate prediction method, conversion rate prediction device, conversion rate prediction apparatus, and storage medium
US20230038310A1 (en) Devices, Methods, and System for Heterogeneous Data-Adaptive Federated Learning
CN112016524A (en) Model training method, face recognition device, face recognition equipment and medium
CN111783951A (en) Model obtaining method, device, equipment and storage medium based on hyper network
CN111177479A (en) Method and device for acquiring feature vectors of nodes in relational network graph
CN112291559B (en) Video coding method, device, equipment and medium
CN115545215A (en) Decentralized federal cluster learning method, decentralized federal cluster learning device, decentralized federal cluster learning equipment and decentralized federal cluster learning medium
CN112735601B (en) Test method, device and equipment for determining infectious disease close-contact population by using Bluetooth
CN111340222B (en) Neural network model searching method and device and electronic equipment
CN111488972B (en) Data migration method, device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant