CN117077817A - Personalized federal learning model training method and device based on label distribution - Google Patents
Personalized federal learning model training method and device based on label distribution Download PDFInfo
- Publication number
- CN117077817A CN117077817A CN202311328295.0A CN202311328295A CN117077817A CN 117077817 A CN117077817 A CN 117077817A CN 202311328295 A CN202311328295 A CN 202311328295A CN 117077817 A CN117077817 A CN 117077817A
- Authority
- CN
- China
- Prior art keywords
- client
- model
- parameters
- trained
- clients
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000009826 distribution Methods 0.000 title claims abstract description 105
- 238000012549 training Methods 0.000 title claims abstract description 93
- 238000000034 method Methods 0.000 title claims abstract description 69
- 238000003860 storage Methods 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 16
- 230000004927 fusion Effects 0.000 claims description 3
- 238000003062 neural network model Methods 0.000 description 16
- 238000010586 diagram Methods 0.000 description 14
- 230000006870 function Effects 0.000 description 14
- 230000008569 process Effects 0.000 description 10
- 230000006872 improvement Effects 0.000 description 8
- 238000012545 processing Methods 0.000 description 6
- 206010028980 Neoplasm Diseases 0.000 description 5
- 201000011510 cancer Diseases 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000003745 diagnosis Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 201000010099 disease Diseases 0.000 description 2
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 230000003902 lesion Effects 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 229920001296 polysiloxane Polymers 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 239000010979 ruby Substances 0.000 description 1
- 229910001750 ruby Inorganic materials 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/098—Distributed learning, e.g. federated learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Medical Informatics (AREA)
- Probability & Statistics with Applications (AREA)
- Image Analysis (AREA)
Abstract
The specification discloses a personalized federal learning model training method and device based on label distribution. The task execution method comprises the following steps: according to the obtained initialization model parameters of the target model, sending the initialization model parameters to each client so that the client deploys the model to be trained locally, training the model to be trained through the local data of the client, obtaining model parameters after the training and updating of each client, and obtaining the label distribution of the local data used by each client when training the respective model to be trained, so as to obtain the client cluster corresponding to each client. And aiming at each client, fusing updated model parameters sent by each client in a client cluster corresponding to the client, and sending the fused parameters to the client so that the client can update parameters of a locally deployed model to be trained according to the fused parameters to execute a target task through the updated model.
Description
Technical Field
The specification relates to the field of artificial intelligence, in particular to a personalized federal learning model training method and device based on label distribution.
Background
With the rapid development of big data and mobile internet, neural network models are applied in various fields, wherein a complex deep neural network model needs a large amount of data to train, and federal learning is generated due to the data privacy protection. When the neural network model is trained, the model can be trained by adopting a federal learning mode under the condition of not sharing the original data, so that the privacy of individual data is protected.
However, in practical application, local data of each client is different in distribution, which results in that the output result of the neural network model trained under the federal learning method is often inaccurate and is not necessarily suitable for the actual service scenarios faced by all clients.
Based on the method, how to improve the accuracy of neural network model training and solve the problem of data heterogeneity generated in federal learning so as to obtain a personalized model which is more fit with each client is a problem to be solved urgently.
Disclosure of Invention
The specification provides a personalized federal learning model training method and device based on label distribution, so as to partially solve the problems existing in the prior art.
The technical scheme adopted in the specification is as follows:
the specification provides a personalized federal learning model training method based on label distribution, comprising the following steps:
the method comprises the steps that a server obtains initialized model parameters of a target model;
for each client, sending the initialized model parameters to the client so that the client deploys a model to be trained locally on the client according to the initialized model parameters, and trains the model to be trained through local data of the client to obtain updated model parameters;
the method comprises the steps of obtaining updated model parameters uploaded by clients and tag distribution of local data used by each client when training a respective model to be trained;
for each client, based on the label distribution, clustering the clients by taking the client as a clustering center to obtain a client cluster corresponding to the client;
and aiming at each client, fusing updated model parameters sent by each client in the client cluster, and sending the fused parameters to the client so that the client updates parameters of a locally deployed model to be trained according to the fused parameters to execute a target task through the updated model.
Optionally, for each client, based on the tag distribution, clustering the clients by using the client as a clustering center to obtain a client cluster corresponding to the client, which specifically includes:
for each client, based on the tag distribution, respectively determining the similarity between the client and local data used by other clients;
and clustering the clients according to the determined similarity between the client and the local data used by other clients to obtain a client cluster taking the client as a clustering center.
Optionally, clustering the clients according to the determined similarity between the local data used by the client and other clients to obtain a client cluster using the client as a cluster center, which specifically includes:
determining a similarity threshold corresponding to the current model training round according to the current model training round, wherein if the current model training round is larger, the similarity threshold corresponding to the current model training round is smaller;
and for any other client except the client, if the similarity between the client and the local data used by the other client is smaller than the similarity threshold, clustering the other client into a client cluster corresponding to the client.
Optionally, the target task includes:
identifying the medical image data through the updated model to output an image identification result;
for each client, the local data of the client includes: medical image data local to the client.
The specification provides a personalized federal learning model training method based on label distribution, comprising the following steps:
the client acquires initialization model parameters aiming at the target model and issued by the server;
the model to be trained is deployed locally through the initialized model parameters, and is trained through local data, so that updated model parameters are obtained;
the updated model parameters and label distribution of local data used when training a locally deployed model to be trained are sent to a server, so that the server clusters according to the label distribution uploaded by the client and label distribution uploaded by other clients to obtain a client cluster taking the client as a clustering center, the client cluster is used as a client cluster corresponding to the client, and the updated model parameters sent by all the clients contained in the client cluster corresponding to the client are fused to send the obtained fused parameters to the client;
And acquiring the fused parameters issued by the server, and updating parameters of the locally deployed model to be trained according to the fused parameters so as to execute the target task through the updated model.
The specification provides a personalized federal learning model training device based on label distribution, comprising:
the initial module is used for acquiring initial model parameters of the target model;
the sending module is used for sending the initialization model parameters to each client so that the client can locally deploy a model to be trained on the client according to the initialization model parameters, and train the model to be trained through local data of the client to obtain updated model parameters;
the acquisition module is used for acquiring updated model parameters uploaded by each client and label distribution of local data used by each client when training a respective model to be trained;
the clustering module is used for clustering the clients by taking the clients as a clustering center based on the label distribution for each client to obtain a client cluster corresponding to the client;
and the fusion module is used for fusing updated model parameters sent by each client in the client cluster aiming at each client, and sending the fused parameters to the client so that the client can update parameters of a locally deployed model to be trained according to the fused parameters to execute a target task through the updated model.
Optionally, the clustering module is specifically configured to, for each client, determine, based on the tag distribution, a similarity between local data used by the client and other clients; and clustering the clients according to the determined similarity between the client and the local data used by other clients to obtain a client cluster taking the client as a clustering center.
The specification provides a personalized federal learning model training device based on label distribution, comprising:
the acquisition module acquires initialization model parameters aiming at the target model and issued by the server;
the training module is used for locally deploying a model to be trained through the initialized model parameters and training the model to be trained through local data so as to obtain updated model parameters;
the sending module is used for sending the updated model parameters and label distribution of local data used when a locally deployed model to be trained is trained to a server, so that the server clusters according to the label distribution uploaded by the device and label distribution uploaded by other clients to obtain a client cluster taking the device as a clustering center, the client cluster is used as a client cluster corresponding to the device, and the updated model parameters sent by all clients contained in the client cluster corresponding to the device are fused to send the obtained fused parameters to the device;
And the updating module is used for acquiring the fused parameters issued by the server, and updating parameters of the locally deployed model to be trained according to the fused parameters so as to execute the target task through the updated model.
The present specification provides a computer readable storage medium storing a computer program which when executed by a processor implements the method of personalized federal learning model training based on tag distribution described above.
The present specification provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of personalized federal learning model training based on tag distribution described above when executing the program.
The above-mentioned at least one technical scheme that this specification adopted can reach following beneficial effect:
according to the personalized federal learning model training method based on label distribution, according to the acquired initialization model parameters of the target model, the initialization model parameters are sent to each client, so that the client deploys the model to be trained locally, the model to be trained is trained through the local data of the client, model parameters after each client is trained and updated are acquired, and label distribution of the local data used by each client when each client trains the respective model to be trained is acquired, so that a client cluster corresponding to each client is obtained. And aiming at each client, fusing updated model parameters sent by each client in a client cluster corresponding to the client, and sending the fused parameters to the client so that the client can update parameters of a locally deployed model to be trained according to the fused parameters to execute a target task through the updated model.
According to the method, in the personalized federal learning model training method based on label distribution, the server clusters the clients with similar label distribution according to the updated parameters after training the model to be trained and the label distribution of the local data used when training the model to be trained, which are uploaded by each client, and fuses the model parameters in each obtained client cluster, so that a neural network model with more accurate output results is obtained, the obtained neural network model is more in line with actual service scenes faced by all clients, the problem of data heterogeneity generated in federal learning is solved, and the personalized neural network model more in line with each client is obtained.
Drawings
The accompanying drawings, which are included to provide a further understanding of the specification, illustrate and explain the exemplary embodiments of the present specification and their description, are not intended to limit the specification unduly. In the drawings:
fig. 1 is a schematic flow chart of a personalized federal learning model training method based on label distribution provided in the present specification;
FIG. 2 is a schematic diagram of a clustering and issuing model parameters based on label distribution provided in the present specification;
FIG. 3 is a schematic flow chart of a personalized federal learning model training method based on label distribution provided in the present specification;
FIG. 4 is a schematic diagram of a personalized federal learning model training device based on label distribution provided in the present specification;
FIG. 5 is a schematic diagram of a personalized federal learning model training device based on label distribution provided in the present specification;
fig. 6 is a schematic diagram of an electronic device corresponding to fig. 1 provided in the present specification.
Detailed Description
For the purposes of making the objects, technical solutions and advantages of the present specification more apparent, the technical solutions of the present specification will be clearly and completely described below with reference to specific embodiments of the present specification and corresponding drawings. It will be apparent that the described embodiments are only some, but not all, of the embodiments of the present specification. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are intended to be within the scope of the present disclosure.
The following describes in detail the technical solutions provided by the embodiments of the present specification with reference to the accompanying drawings.
Fig. 1 is a schematic flow chart of a personalized federal learning model training method based on label distribution provided in the present specification, including the following steps:
s101: the server obtains the initialized model parameters of the target model.
Federal learning is a machine learning method that aims to have multiple participants co-train a machine learning model without the need to concentrate the original data set in one place. The basic principle of federal learning is to update and aggregate global models between participants in an iterative manner to achieve the goal of co-learning.
However, in federation learning, the distribution of local data of the client under the real scene is often different, which results in that the result output by the neural network model obtained by adopting the federation learning algorithm is often inaccurate, and cannot conform to the actual service scene faced by all the clients, even unlike the model that the client uses the local data to train out independently, a new scheme is needed to solve the problem of data heterogeneity in federation learning, so as to obtain the personalized neural network model more conforming to each client.
Based on the above, the specification provides a personalized federal learning model training method based on label distribution, which calculates the similarity of label distribution of local data of each client to obtain clients with similar local data label distribution, and for each client, clusters the clients by using the client as a cluster center to obtain a client cluster by using the client as a cluster center, fuses model parameters of each client contained in the client cluster, and sends the fused parameters to the client, so that the client updates parameters of a model to be trained locally deployed by the client according to the fused parameters, and executes a target task through the updated model. According to the method, the neural network model with more accurate output results can be obtained under the condition that the label distribution of local data of each client is different, the problem of data heterogeneity caused by federal learning is solved, the training effect of the neural network model is ensured, and the more personalized neural network model is obtained.
In the personalized federal learning model training method based on label distribution, a server firstly needs to assign initial values to parameters (such as network weights) in a target model, wherein various methods for initializing model parameters exist, for example, a random initialization mode is adopted to randomly initialize the parameters; for another example, model parameters obtained by training the target model in a pre-training manner are used as the initialization model parameters of the obtained target model.
S102: and sending the initialization model parameters to each client so that the client deploys a model to be trained locally on the client according to the initialization model parameters, and trains the model to be trained through local data of the client to obtain updated model parameters.
In the specification, for each client, the server sends the initialized model parameters to the client, so that the client deploys a local model to be trained according to the model parameters sent by the server, and trains the model to be trained based on the local data of the client. The client may train the locally deployed model to be trained by using various algorithms, for example, a random gradient descent method (Stochastic Gradient Descent, SGD) is used to update model parameters of the locally deployed model to be trained, where SGD is an optimization algorithm used for parameter learning in neural network training.
S103: and acquiring updated model parameters uploaded by each client and label distribution of local data used by each client when training a respective model to be trained.
In practical applications, there is often a difference in local data used by each client, and if a unified model is trained by ignoring the difference in local data of each client, it may result in that when any client executes a task through the trained model, a service result that is inconsistent with the actual service situation faced by itself may be obtained.
For example, if in a medical scenario, each client represents a client used by a medical institution participating in model training, the patient in each medical institution often has different data distribution of the patient information and the diagnosis result that remain due to differences in age, movement frequency, lifestyle, and the like. Assuming that a medical institution is mainly directed to cancer screening for men, the data used may be mostly medical image data of men, and if a unified medical image recognition model is trained by using medical image data of patients used in all medical institutions in a traditional federal learning manner, the use of a large amount of medical image data of women in a model training stage introduces recognition noise to the unified medical image recognition model deployed by the medical institution, thereby reducing the diagnostic accuracy of the medical institution for cancer screening for men.
In order to avoid the above problems, in the present specification, the server needs to obtain updated model parameters uploaded by each client, and tag distribution of local data used when each client trains a respective model to be trainedWhere i in the formula here represents client i.
The tag distribution mentioned above is mainly used to represent the duty ratio of various types of data in the local data used by one client. Continuing with the above example, for a client used by a medical facility, the tag distribution of the local data used by the client can be used to represent the duty cycle of the medical image data for men and the duty cycle of the medical image data for women.
Of course, the tag distribution mentioned above may also be used to represent the duty ratio of various recognition results obtained after the client recognizes the local data. Continuing with the above example, for a client used by a medical institution, the tag distribution of the local data used by the client may also be used to represent the duty ratio of the medical image data including the cancer cell image in the medical image data, and the duty ratio of the medical image data not including the cancer cell image in the medical image data.
It should be noted that, the tag distribution of the local data used when each client trains the respective model to be trained only needs to be sent once, and the tag distribution can be maintained in the server, and the local data of the client does not need to be uploaded, thereby protecting the data privacy of the client.
S104: and for each client, based on the label distribution, clustering the clients by taking the clients as a clustering center to obtain client clusters corresponding to the clients.
In this specification, the server determines the distance between the tag distributions of the local data of each client based on the tag distributions of the local data used by each client to train the respective model to be trained. Specifically, when calculating the distance between the tag distributions of two client local data, it is first assumed that the tag distributions of client i and client j local data are respectivelyAnd->ThenThe distance between them can be expressed by the following formula:
wherein, for the client i, the server may calculate the distance between the client i and the tag distribution of each other client local data, to obtain a set of data, where m represents the total number of clients participating in model training:
The calculation formula of the client i is only written, and similarly, for any two clients, the server can determine the similarity between the two client local data according to the distance between the tag distributions of the two client local data. The smaller the distance between the tag distribution of the local data of the two clients, the higher the similarity between the local data used by the two clients to train the model to be trained.
Further, in the present specification, a subtraction function may be set in the server，/>To train on model->Similarity threshold corresponding to training rounds, similarity threshold +.>Values and iteration turns ∈ ->And negative correlation is formed between the model training sequences, specifically, if the current model training sequence is larger, the similarity threshold corresponding to the current model training sequence is smaller.
Then, for any two clients, the server may determine the similarity between the local data of the two clients and the similarity threshold by comparing the similarity with the local data of the two clientsTo cluster clients. Wherein if the similarity between the local data used by the two clients is less than the similarity threshold +.>The two clients may be clustered into one client cluster. For example, the server may determine the similarity of the local data between the client i and all other clients participating in model training, and then determine the obtained similarity and the similarity threshold +. >By contrast, a similar client set +.>。
To summarize, ifThen client->. That is, in the t training round, the similarity between the local data of the client j and the local data of the client i is smaller than the similarity threshold, and the model parameters of the client j and the model parameters of the client i can help to complete personalized model training with each other. According to the above mode, the server can cluster the clients to obtain the similarity set +.>The set may also be considered as a cluster of clients clustered by the server.
It should be noted that the foregoing is mainly used to describe how to determine the similarity between the local data used by two clients, but in this specification, it is actually necessary to determine, for each client, a client cluster in which the client is a cluster center. That is, for any client, the server needs to calculate the similarity between the other clients and the local data used by the client, so as to determine which local data used by the other clients are highly similar to the local data used by the client, and further determine the client cluster using the client as the cluster center. In the subsequent process, the fused parameters obtained through the client cluster are only issued to the client, and are not issued to other clients contained in the client cluster.
That is, when determining the fused parameters corresponding to the client, other clients included in the client cluster corresponding to the client mainly play an auxiliary role, and the purpose of the method is to determine the fused parameters to be used by the client, as shown in fig. 2.
Fig. 2 is a schematic diagram of a clustering and model parameter issuing process based on label distribution provided in the present specification.
The client A, B, C, D, E is assumed to use the client a as a cluster center, so that other clients with similar label distribution to the client a are obtained, namely, a client B and a client C, respectively, to form a client cluster using the client a as the cluster center, wherein the client cluster includes clients A, B and C. The server fuses the model parameters of all clients in the client cluster, and then only sends the fused parameters to the client A.
And for the client B, the client B is taken as a clustering center, so that a client C with similar label distribution to the client B is obtained, a client cluster taking the client B as the clustering center is formed, and the client cluster comprises the client B and the client C. The server fuses the model parameters of all clients in the client cluster, and then only transmits the fused parameters to the client B.
And for the client C, the client C is taken as a clustering center, a client D and a client E which have similar label distribution with the client C are obtained, a client cluster taking the client C as the clustering center is formed, and the client cluster comprises clients C, D and E. The server fuses the model parameters of all clients in the client cluster, and then only sends the fused parameters to the client C.
And for the client D, taking the client D as a clustering center to obtain other clients which have similar label distribution with the client D, namely a client C and a client E, and forming a client cluster taking the client D as the clustering center, wherein the client cluster comprises clients C, D and E. The server fuses the model parameters of all clients in the client cluster, and then only sends the fused parameters to the client D.
And for the client E, taking the client E as a clustering center to obtain other clients with similar label distribution as the client E, namely a client A and a client C, and forming a client cluster taking the client E as the clustering center, wherein the client cluster comprises clients A, C and E. The server fuses the model parameters of all clients in the client cluster, and then only sends the fused parameters to the client E.
S105: and aiming at each client, fusing updated model parameters sent by each client in the client cluster, and sending the fused parameters to the client so that the client updates parameters of a locally deployed model to be trained according to the fused parameters to execute a target task through the updated model.
And for each client, clustering the clients by taking the client as a clustering center to obtain a client cluster by taking the client as the clustering center, wherein the server can fuse updated model parameters uploaded by the clients contained in the client cluster for each client. In particular, the server needs to be able to use a parameter averaging approach toThe updated model parameters uploaded by the clients are fused, and the following formula can be referred to specifically:
wherein,representing model parameters obtained by fusing model parameters of all clients contained in the client cluster in the t-th training. />T in (a) represents the t-th training round in model training, k represents the kth client in the client cluster, so +.>Meaning model parameters of the kth client in the cluster of clients in the t-th training round.
After the fused parameters are obtained, the server can send the fused parameters to the client, and update parameters of the model to be trained which is deployed locally by the client so as to execute the target task through the updated model.
For any client, after obtaining the fused parameters, each client updates the local model parameters by using the fused parameters to obtain an updated model, and then each client continues to train the updated model according to the local data and a preset loss function.
In this specification, the loss function adopted by the client is specifically expressed by the following formula:
in the above-mentioned formula(s),for client cluster->（/>For representing the loss function of the kth client in the cluster of clients obtained with the client i as cluster center in the t-th training round). From this loss function, it is actually desirable that the sum of the loss values generated by the models deployed by all clients in a client cluster during the training process is as minimum as possible, so that the locally deployed models of the clients included in the client cluster can reach a round of training goal. After the client trains the locally deployed model in the above manner, updated model parameters obtained through model training are returned to the server, and the server continues to fuse the updated model parameters in the above manner.
According to the process, as the training rounds are continuously iterated, the clustering precision of clustering the clients is continuously improved (namely, as the training rounds are increased, the number of the clients contained in one client cluster is continuously reduced, but local data used by the clients clustered in one cluster are increasingly close in label distribution), so that the finally determined fused parameters of one client by the server are customized for the client based on training data used by other clients which are highly similar to the client in an actual service scene, the individuation of a locally deployed model of the client is reflected to a certain extent, the locally deployed model of the client can be more consistent with the actual service scene of the client, and the service accuracy of the client is improved.
The trained local model can be used for executing a target task, and the target task can be to continue training the model according to the method so as to finally obtain a trained model through continuous iteration. Of course, the target task may also refer to other forms of tasks, for example, if the model is applied in a medical scenario, the trained model may perform a disease diagnosis target task, for example, medical image data of a patient may be input into an updated model, and the updated model may output an image recognition result, such as a recognition result for a cancer lesion tissue.
In the specification, for each client, based on the label distribution, the server side clusters the clients by using the clients as a cluster center to obtain client clusters corresponding to the clients, fuses model parameters uploaded by all clients in any client cluster, and sends the fused model parameters to the clients so that the clients can continuously execute target tasks according to the fused model parameters. According to the method, under the condition that the label distribution of local data of each client is different, each client can obtain a neural network model with more accurate output results, the obtained neural network model is more in line with actual service scenes faced by all clients, the problem of data heterogeneity generated in federal learning is solved, and a personalized neural network model which is more in line with each client is obtained.
While the description has been made with the server as the execution subject, the client in the present specification is improved for the conventional federal learning, and the description will be made with the client as the execution subject.
Fig. 3 is a schematic flow chart of a personalized federal learning model training method based on label distribution provided in the present specification.
S301: the client acquires initialization model parameters aiming at the target model and issued by the server.
The client first needs to obtain the initialization model parameters for the target model issued by the server, where the initialization of the model parameters by the server is specifically described in S101 above, and will not be described here.
S302: and deploying the model to be trained locally through the initialized model parameters, and training the model to be trained through local data to obtain updated model parameters.
The client deploys the model to be trained locally according to the initialized model parameters issued by the server, and trains the local model to be trained according to the local data, wherein the model can be trained by adopting methods such as random gradient descent and the like so as to update the model parameters. The specific process is described in S102 above, and will not be described here again.
S303: and transmitting the updated model parameters and label distribution of local data used when training a locally deployed model to be trained to a server, so that the server clusters according to the label distribution uploaded by the client and label distribution uploaded by other clients to obtain a client cluster taking the client as a clustering center, wherein the client cluster is used as a client cluster corresponding to the client, and fusing the updated model parameters transmitted by each client contained in the client cluster corresponding to the client to transmit the obtained fused parameters to the client.
The client sends the updated model parameters to the server, and the label distribution of the local data used when training the locally deployed model to be trained is sent to the server, wherein the label distribution of the local data used when each client trains the respective model to be trained only needs to be sent once. Next, for any client, the server needs to cluster the clients by taking the client as a cluster center according to label distribution sent by all clients participating in training, form a client cluster by taking the client as the cluster center, and then fuse model parameters of the clients contained in the client cluster to obtain fused parameters. The specific procedures have been specifically described in the above-mentioned S103, S104 and S105, and will not be described here.
S304: and acquiring the fused parameters issued by the server, and updating parameters of the locally deployed model to be trained according to the fused parameters so as to execute the target task through the updated model.
The client acquires the model parameters fused by the server, trains the locally deployed model to be trained according to the fused parameters so as to update the parameters of the locally deployed model to be trained, and the updated model continues to execute the target task. The target task may be to continue training the model according to the above method, so as to obtain a trained model finally through continuous iteration; the target task may also refer to other forms of tasks, for example, if the model is applied in a medical scenario, the trained model may perform a disease diagnosis target task.
The foregoing describes one or more embodiments of the present disclosure implementing a personalized federal learning model training method based on label distribution, and based on the same concept, the present disclosure further provides a corresponding personalized federal learning model training device based on label distribution, as shown in fig. 4 and fig. 5.
Fig. 4 is a schematic diagram of a personalized federal learning model training device based on label distribution provided in the present specification, including:
an initialization module 401, configured to obtain initialization model parameters of a target model;
the sending module 402 is configured to send the initialization model parameter to each client, so that the client deploys a model to be trained locally on the client according to the initialization model parameter, and trains the model to be trained according to local data of the client to obtain updated model parameters;
the obtaining module 403 is configured to obtain updated model parameters uploaded by each client, and tag distribution of local data used when each client trains a respective model to be trained;
the clustering module 404 is configured to, for each client, perform client clustering with the client as a clustering center based on the tag distribution, to obtain a client cluster corresponding to the client;
And the fusion module 405 fuses the updated model parameters sent by each client included in the client cluster for each client, and sends the fused parameters to the client, so that the client updates parameters of the locally deployed model to be trained according to the fused parameters, and the updated model is used for executing the target task.
Optionally, the clustering module 404 is specifically configured to, for each client, determine, based on the tag distribution, a similarity between the local data used by the client and other clients; and clustering the clients according to the determined similarity between the client and the local data used by other clients to obtain a client cluster taking the client as a clustering center.
Optionally, the clustering module 404 is specifically configured to determine, according to the current model training round, a similarity threshold corresponding to the current model training round, where if the current model training round is larger, the similarity threshold corresponding to the current model training round is smaller; and for any other client except the client, if the similarity between the client and the local data used by the other client is smaller than the similarity threshold, clustering the other client into a client cluster corresponding to the client.
Optionally, the target task includes: identifying the medical image data through the updated model to output an image identification result; for each client, the local data of the client includes: medical image data local to the client.
Fig. 5 is a schematic diagram of a personalized federal learning model training device based on label distribution provided in the present specification, including:
the obtaining module 501 is configured to obtain an initialized model parameter for a target model, which is issued by a server;
the training module 502 is configured to deploy a model to be trained locally through the initialized model parameters, and train the model to be trained through local data, so as to obtain updated model parameters;
a sending module 503, configured to send the updated model parameters and tag distributions of local data used when training a locally deployed model to be trained to a server, so that the server clusters according to the tag distributions uploaded by the device and tag distributions uploaded by other clients, obtains a client cluster using the device as a cluster center, and fuses the updated model parameters sent by each client included in the client cluster corresponding to the device, so as to send the obtained fused parameters to the device;
And the updating module 504 is configured to obtain the fused parameters issued by the server, and update parameters of the locally deployed model to be trained according to the fused parameters, so as to execute the target task through the updated model.
The present specification also provides a computer readable storage medium storing a computer program operable to perform a personalized federal learning model training method based on tag distribution as provided in fig. 1 above.
The present specification also provides a schematic structural diagram of an electronic device corresponding to fig. 1 shown in fig. 6. At the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile storage, as illustrated in fig. 6, although other hardware required by other services may be included. The processor reads the corresponding computer program from the nonvolatile memory into the memory and then runs the computer program to realize the personalized federal learning model training method based on the label distribution, which is described in the above-mentioned figure 1. Of course, other implementations, such as logic devices or combinations of hardware and software, are not excluded from the present description, that is, the execution subject of the following processing flows is not limited to each logic unit, but may be hardware or logic devices.
Improvements to one technology can clearly distinguish between improvements in hardware (e.g., improvements to circuit structures such as diodes, transistors, switches, etc.) and software (improvements to the process flow). However, with the development of technology, many improvements of the current method flows can be regarded as direct improvements of hardware circuit structures. Designers almost always obtain corresponding hardware circuit structures by programming improved method flows into hardware circuits. Therefore, an improvement of a method flow cannot be said to be realized by a hardware entity module. For example, a programmable logic device (Programmable Logic Device, PLD) (e.g., field programmable gate array (Field Programmable Gate Array, FPGA)) is an integrated circuit whose logic function is determined by the programming of the device by a user. A designer programs to "integrate" a digital system onto a PLD without requiring the chip manufacturer to design and fabricate application-specific integrated circuit chips. Moreover, nowadays, instead of manually manufacturing integrated circuit chips, such programming is mostly implemented by using "logic compiler" software, which is similar to the software compiler used in program development and writing, and the original code before the compiling is also written in a specific programming language, which is called hardware description language (Hardware Description Language, HDL), but not just one of the hdds, but a plurality of kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), lava, lola, myHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog are currently most commonly used. It will also be apparent to those skilled in the art that a hardware circuit implementing the logic method flow can be readily obtained by merely slightly programming the method flow into an integrated circuit using several of the hardware description languages described above.
The controller may be implemented in any suitable manner, for example, the controller may take the form of, for example, a microprocessor or processor and a computer readable medium storing computer readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable logic controllers, and embedded microcontrollers, examples of which include, but are not limited to, the following microcontrollers: ARC 625D, atmel AT91SAM, microchip PIC18F26K20, and Silicone Labs C8051F320, the memory controller may also be implemented as part of the control logic of the memory. Those skilled in the art will also appreciate that, in addition to implementing the controller in a pure computer readable program code, it is well possible to implement the same functionality by logically programming the method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. Such a controller may thus be regarded as a kind of hardware component, and means for performing various functions included therein may also be regarded as structures within the hardware component. Or even means for achieving the various functions may be regarded as either software modules implementing the methods or structures within hardware components.
The system, apparatus, module or unit set forth in the above embodiments may be implemented in particular by a computer chip or entity, or by a product having a certain function. One typical implementation is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
For convenience of description, the above devices are described as being functionally divided into various units, respectively. Of course, the functions of each element may be implemented in one or more software and/or hardware elements when implemented in the present specification.
It will be appreciated by those skilled in the art that embodiments of the present description may be provided as a method, system, or computer program product. Accordingly, the present specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present description can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
The present description is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the specification. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
It will be appreciated by those skilled in the art that embodiments of the present description may be provided as a method, system, or computer program product. Accordingly, the present specification may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present description can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
The description may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.
The foregoing is merely exemplary of the present disclosure and is not intended to limit the disclosure. Various modifications and alterations to this specification will become apparent to those skilled in the art. Any modifications, equivalent substitutions, improvements, or the like, which are within the spirit and principles of the present description, are intended to be included within the scope of the claims of the present description.
Claims (10)
1. A personalized federal learning model training method based on label distribution is characterized by comprising the following steps:
the method comprises the steps that a server obtains initialized model parameters of a target model;
for each client, sending the initialized model parameters to the client so that the client deploys a model to be trained locally on the client according to the initialized model parameters, and trains the model to be trained through local data of the client to obtain updated model parameters;
the method comprises the steps of obtaining updated model parameters uploaded by clients and tag distribution of local data used by each client when training a respective model to be trained;
for each client, based on the label distribution, clustering the clients by taking the client as a clustering center to obtain a client cluster corresponding to the client;
and aiming at each client, fusing updated model parameters sent by each client in the client cluster, and sending the fused parameters to the client so that the client updates parameters of a locally deployed model to be trained according to the fused parameters to execute a target task through the updated model.
2. The method of claim 1, wherein for each client, based on the tag distribution, clustering the clients with the client as a cluster center to obtain a client cluster corresponding to the client, specifically comprising:
for each client, based on the tag distribution, respectively determining the similarity between the client and local data used by other clients;
and clustering the clients according to the determined similarity between the client and the local data used by other clients to obtain a client cluster taking the client as a clustering center.
3. The method of claim 2, wherein clustering the clients according to the determined similarity between the local data used by the client and other clients to obtain a client cluster with the client as a cluster center, specifically comprising:
determining a similarity threshold corresponding to the current model training round according to the current model training round, wherein if the current model training round is larger, the similarity threshold corresponding to the current model training round is smaller;
and for any other client except the client, if the similarity between the client and the local data used by the other client is smaller than the similarity threshold, clustering the other client into a client cluster corresponding to the client.
4. A method according to any one of claims 1 to 3, wherein the target task comprises: identifying the medical image data through the updated model to output an image identification result;
for each client, the local data of the client includes: medical image data local to the client.
5. A personalized federal learning model training method based on label distribution is characterized by comprising the following steps:
the client acquires initialization model parameters aiming at the target model and issued by the server;
the model to be trained is deployed locally through the initialized model parameters, and is trained through local data, so that updated model parameters are obtained;
the updated model parameters and label distribution of local data used when training a locally deployed model to be trained are sent to a server, so that the server clusters according to the label distribution uploaded by the client and label distribution uploaded by other clients to obtain a client cluster taking the client as a clustering center, the client cluster is used as a client cluster corresponding to the client, and the updated model parameters sent by all the clients contained in the client cluster corresponding to the client are fused to send the obtained fused parameters to the client;
And acquiring the fused parameters issued by the server, and updating parameters of the locally deployed model to be trained according to the fused parameters so as to execute the target task through the updated model.
6. Personalized federal learning model training device based on label distribution, which is characterized by comprising:
the initial module is used for acquiring initial model parameters of the target model;
the sending module is used for sending the initialization model parameters to each client so that the client can locally deploy a model to be trained on the client according to the initialization model parameters, and train the model to be trained through local data of the client to obtain updated model parameters;
the acquisition module is used for acquiring updated model parameters uploaded by each client and label distribution of local data used by each client when training a respective model to be trained;
the clustering module is used for clustering the clients by taking the clients as a clustering center based on the label distribution for each client to obtain a client cluster corresponding to the client;
and the fusion module is used for fusing updated model parameters sent by each client in the client cluster aiming at each client, and sending the fused parameters to the client so that the client can update parameters of a locally deployed model to be trained according to the fused parameters to execute a target task through the updated model.
7. The apparatus of claim 6, wherein the clustering module is specifically configured to, for each client, determine, based on the tag distribution, a similarity between the client and local data used by other clients, respectively; and clustering the clients according to the determined similarity between the client and the local data used by other clients to obtain a client cluster taking the client as a clustering center.
8. Personalized federal learning model training device based on label distribution, which is characterized by comprising:
the acquisition module acquires initialization model parameters aiming at the target model and issued by the server;
the training module is used for locally deploying a model to be trained through the initialized model parameters and training the model to be trained through local data so as to obtain updated model parameters;
the sending module is used for sending the updated model parameters and label distribution of local data used when a locally deployed model to be trained is trained to a server, so that the server clusters according to the label distribution uploaded by the device and label distribution uploaded by other clients to obtain a client cluster taking the device as a clustering center, the client cluster is used as a client cluster corresponding to the device, and the updated model parameters sent by all clients contained in the client cluster corresponding to the device are fused to send the obtained fused parameters to the device;
And the updating module is used for acquiring the fused parameters issued by the server, and updating parameters of the locally deployed model to be trained according to the fused parameters so as to execute the target task through the updated model.
9. A computer readable storage medium, characterized in that the storage medium stores a computer program which, when executed by a processor, implements the method of any of the preceding claims 1-5.
10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of any of the preceding claims 1-5 when executing the program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311328295.0A CN117077817B (en) | 2023-10-13 | 2023-10-13 | Personalized federal learning model training method and device based on label distribution |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311328295.0A CN117077817B (en) | 2023-10-13 | 2023-10-13 | Personalized federal learning model training method and device based on label distribution |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117077817A true CN117077817A (en) | 2023-11-17 |
| CN117077817B CN117077817B (en) | 2024-01-30 |
Family
ID=88713668
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311328295.0A Active CN117077817B (en) | 2023-10-13 | 2023-10-13 | Personalized federal learning model training method and device based on label distribution |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117077817B (en) |
Citations (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113033820A (en) * | 2021-03-25 | 2021-06-25 | 支付宝(杭州)信息技术有限公司 | Federal learning method, device and equipment |
| CN113487351A (en) * | 2021-07-05 | 2021-10-08 | 哈尔滨工业大学（深圳） | Privacy protection advertisement click rate prediction method, device, server and storage medium |
| CN113988314A (en) * | 2021-11-09 | 2022-01-28 | 长春理工大学 | Cluster federal learning method and system for selecting client |
| CN114186237A (en) * | 2021-10-26 | 2022-03-15 | 北京理工大学 | Truth-value discovery-based robust federated learning model aggregation method |
| CN114357067A (en) * | 2021-12-15 | 2022-04-15 | 华南理工大学 | Personalized federal meta-learning method for data isomerism |
| WO2022105022A1 (en) * | 2020-11-18 | 2022-05-27 | 杭州趣链科技有限公司 | Federated learning-based machine learning method, electronic device and storage medium |
| WO2022111639A1 (en) * | 2020-11-30 | 2022-06-02 | 华为技术有限公司 | Federated learning method and apparatus, device, system, and computer-readable storage medium |
| CN114758784A (en) * | 2022-03-29 | 2022-07-15 | 南京理工大学 | Method for distributing weight of participants in federal learning based on clustering algorithm |
| WO2022162677A1 (en) * | 2021-01-29 | 2022-08-04 | Telefonaktiebolaget Lm Ericsson (Publ) | Distributed machine learning with new labels using heterogeneous label distribution |
| CN115238909A (en) * | 2022-06-28 | 2022-10-25 | 华为技术有限公司 | Data value evaluation method based on federal learning and related equipment thereof |
| CN115238908A (en) * | 2022-03-15 | 2022-10-25 | 华东师范大学 | Data generation method based on variational self-encoder, unsupervised clustering algorithm and federal learning |
| CN115239675A (en) * | 2022-07-29 | 2022-10-25 | 平安科技（深圳）有限公司 | Training method of classification model, image classification method and device, equipment and medium |
| CN115311478A (en) * | 2022-08-16 | 2022-11-08 | 悉科大创新研究(深圳)有限公司 | Federal image classification method based on image depth clustering and storage medium |
| US20220358417A1 (en) * | 2021-04-29 | 2022-11-10 | Zhejiang University | FEDERATED LEARNING METHOD FOR k-means CLUSTERING ALGORITHM |
| CN115329885A (en) * | 2022-08-23 | 2022-11-11 | 浙江工业大学 | Personalized federal learning method and device based on privacy protection |
| CN115344883A (en) * | 2022-06-29 | 2022-11-15 | 上海工程技术大学 | Personalized federal learning method and device for processing unbalanced data |
| CN115577301A (en) * | 2022-09-27 | 2023-01-06 | 华中科技大学 | Personalized splitting learning method and application for distributed system |
| CN115577360A (en) * | 2022-11-14 | 2023-01-06 | 湖南大学 | Gradient-independent clustering federal learning method and system |
| CN115796310A (en) * | 2022-11-08 | 2023-03-14 | 北京百度网讯科技有限公司 | Information recommendation method, information recommendation device, information recommendation model training device, information recommendation equipment and storage medium |
| CN116204599A (en) * | 2023-05-06 | 2023-06-02 | 成都三合力通科技有限公司 | User information analysis system and method based on federal learning |
| CN116471286A (en) * | 2023-03-30 | 2023-07-21 | 上海电信科技发展有限公司 | Internet of things data sharing method based on block chain and federal learning |
| CN116540876A (en) * | 2023-05-09 | 2023-08-04 | 合肥工业大学 | Human body action recognition method based on individual information personalized federal learning |
-
2023
- 2023-10-13 CN CN202311328295.0A patent/CN117077817B/en active Active
Patent Citations (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022105022A1 (en) * | 2020-11-18 | 2022-05-27 | 杭州趣链科技有限公司 | Federated learning-based machine learning method, electronic device and storage medium |
| WO2022111639A1 (en) * | 2020-11-30 | 2022-06-02 | 华为技术有限公司 | Federated learning method and apparatus, device, system, and computer-readable storage medium |
| WO2022162677A1 (en) * | 2021-01-29 | 2022-08-04 | Telefonaktiebolaget Lm Ericsson (Publ) | Distributed machine learning with new labels using heterogeneous label distribution |
| CN113033820A (en) * | 2021-03-25 | 2021-06-25 | 支付宝(杭州)信息技术有限公司 | Federal learning method, device and equipment |
| US20220358417A1 (en) * | 2021-04-29 | 2022-11-10 | Zhejiang University | FEDERATED LEARNING METHOD FOR k-means CLUSTERING ALGORITHM |
| CN113487351A (en) * | 2021-07-05 | 2021-10-08 | 哈尔滨工业大学（深圳） | Privacy protection advertisement click rate prediction method, device, server and storage medium |
| CN114186237A (en) * | 2021-10-26 | 2022-03-15 | 北京理工大学 | Truth-value discovery-based robust federated learning model aggregation method |
| CN113988314A (en) * | 2021-11-09 | 2022-01-28 | 长春理工大学 | Cluster federal learning method and system for selecting client |
| CN114357067A (en) * | 2021-12-15 | 2022-04-15 | 华南理工大学 | Personalized federal meta-learning method for data isomerism |
| CN115238908A (en) * | 2022-03-15 | 2022-10-25 | 华东师范大学 | Data generation method based on variational self-encoder, unsupervised clustering algorithm and federal learning |
| CN114758784A (en) * | 2022-03-29 | 2022-07-15 | 南京理工大学 | Method for distributing weight of participants in federal learning based on clustering algorithm |
| CN115238909A (en) * | 2022-06-28 | 2022-10-25 | 华为技术有限公司 | Data value evaluation method based on federal learning and related equipment thereof |
| CN115344883A (en) * | 2022-06-29 | 2022-11-15 | 上海工程技术大学 | Personalized federal learning method and device for processing unbalanced data |
| CN115239675A (en) * | 2022-07-29 | 2022-10-25 | 平安科技（深圳）有限公司 | Training method of classification model, image classification method and device, equipment and medium |
| CN115311478A (en) * | 2022-08-16 | 2022-11-08 | 悉科大创新研究(深圳)有限公司 | Federal image classification method based on image depth clustering and storage medium |
| CN115329885A (en) * | 2022-08-23 | 2022-11-11 | 浙江工业大学 | Personalized federal learning method and device based on privacy protection |
| CN115577301A (en) * | 2022-09-27 | 2023-01-06 | 华中科技大学 | Personalized splitting learning method and application for distributed system |
| CN115796310A (en) * | 2022-11-08 | 2023-03-14 | 北京百度网讯科技有限公司 | Information recommendation method, information recommendation device, information recommendation model training device, information recommendation equipment and storage medium |
| CN115577360A (en) * | 2022-11-14 | 2023-01-06 | 湖南大学 | Gradient-independent clustering federal learning method and system |
| CN116471286A (en) * | 2023-03-30 | 2023-07-21 | 上海电信科技发展有限公司 | Internet of things data sharing method based on block chain and federal learning |
| CN116204599A (en) * | 2023-05-06 | 2023-06-02 | 成都三合力通科技有限公司 | User information analysis system and method based on federal learning |
| CN116540876A (en) * | 2023-05-09 | 2023-08-04 | 合肥工业大学 | Human body action recognition method based on individual information personalized federal learning |
Non-Patent Citations (4)
| Title |
|---|
| PU TIAN 等: "WSCC: A Weight-Similarity-Based Client Clustering Approach for Non-IID Federated Learning", 《IEEE INTERNET OF THINGS IOURNAL》, vol. 9, no. 20, pages 20243 - 20256, XP011922221, DOI: 10.1109/JIOT.2022.3175149 * |
| SARHAD ARISDAKESSIAN 等: "Towards Instant Clustering Approach for Federated Learning Client Selection", 《2023 INTERNATIONAL CONFERENCE ON COMPUTING, NETWORKING AND COMMUNICATIONS (ICNC)》, pages 409 - 413 * |
| 何杨: "物联网中基于联邦学习的数据共享关键技术研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》, vol. 2023, no. 2, pages 136 - 1293 * |
| 郑美光 等: "基于互信息软聚类的个性化联邦学习算法", 《计算机工程》, vol. 49, no. 8, pages 20 - 28 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117077817B (en) | 2024-01-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112766468B (en) | Trajectory prediction method and device, storage medium and electronic equipment | |
| CN115981870B (en) | Data processing method and device, storage medium and electronic equipment | |
| CN112308113A (en) | Target identification method, device and medium based on semi-supervision | |
| CN115035158B (en) | Target tracking method and device, electronic equipment and storage medium | |
| CN112912889B (en) | Image template updating method, device and storage medium | |
| CN113988162A (en) | Model training and image recognition method and device, storage medium and electronic equipment | |
| CN117077817B (en) | Personalized federal learning model training method and device based on label distribution | |
| CN117409466A (en) | Three-dimensional dynamic expression generation method and device based on multi-label control | |
| CN117635822A (en) | Model training method and device, storage medium and electronic equipment | |
| CN112734851B (en) | Pose determination method and device | |
| CN117876610B (en) | Model training method, device and storage medium for three-dimensional construction model | |
| CN116152246B (en) | Image recognition method, device, equipment and storage medium | |
| CN117726907B (en) | Training method of modeling model, three-dimensional human modeling method and device | |
| CN117870696B (en) | Path navigation method and device based on perception information fusion and electronic equipment | |
| CN116578877B (en) | Method and device for model training and risk identification of secondary optimization marking | |
| CN116563387A (en) | Training method and device of calibration model, storage medium and electronic equipment | |
| CN116246774B (en) | Classification method, device and equipment based on information fusion | |
| CN116996397B (en) | Network packet loss optimization method and device, storage medium and electronic equipment | |
| CN113673601B (en) | Behavior recognition method and device, storage medium and electronic equipment | |
| CN115862675B (en) | Emotion recognition method, device, equipment and storage medium | |
| CN117576522B (en) | Model training method and device based on mimicry structure dynamic defense | |
| CN112149836B (en) | Machine learning program updating method, device and equipment | |
| US20230342623A1 (en) | Machine learning pipeline | |
| CN117057442A (en) | Model training method, device and equipment based on federal multitask learning | |
| CN117132806A (en) | Model training method and device, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |