CN115775025A - Lightweight federated learning method and system for space-time data heterogeneous scene - Google Patents
Lightweight federated learning method and system for space-time data heterogeneous scene Download PDFInfo
- Publication number
- CN115775025A CN115775025A CN202211649919.4A CN202211649919A CN115775025A CN 115775025 A CN115775025 A CN 115775025A CN 202211649919 A CN202211649919 A CN 202211649919A CN 115775025 A CN115775025 A CN 115775025A
- Authority
- CN
- China
- Prior art keywords
- client
- federal
- model
- communication
- global
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Computer And Data Communications (AREA)
Abstract
The invention discloses a lightweight federated learning method and system for a space-time data heterogeneous scene, which can distribute different communication probabilities to clients according to the change condition of data distribution of the clients, so that the clients which have large data distribution change degree and are not perceived by a federated global model participate in federated communication with higher probability instead of equally increasing the communication traffic of all the clients. By inclining the communication resources, the global data distribution change can be sensed by less communication traffic, and the communication efficiency of federal learning under the space-time heterogeneous scene is improved. By using the method and the system provided by the invention, each image acquisition and analysis terminal dynamically adjusts the probability of the image acquisition and analysis terminal participating in the federal communication in each round by using the change information of the private case data distribution and the gradient information of the private detection model of the image acquisition and analysis terminal, thereby achieving the same federal learning performance with less federal communication cost and effectively reducing the federal learning communication traffic.
Description
Technical Field
The invention relates to the technical field of federal learning, in particular to a lightweight federal learning method and system for a space-time data heterogeneous scene.
Background
In recent years, the deep learning technology remarkably improves the performance of tasks such as computer vision, natural language processing and the like. By training a Deep Neural Network (DNN) with million-level parameters on mass data, the performance of a deep learning model gradually surpasses that of traditional machine learning and becomes a mainstream of the artificial intelligence technology industry. In the traditional deep learning, due to the dependence on massive training data, a centralized mode that data are collected at edge equipment and uploaded to a cloud server to perform model training and reasoning service is generally adopted.
However, in recent years, with the gradual emergence of the act of data security and personal privacy protection, the process of uploading private data collected from a user device such as a mobile phone or a tablet to a central server is limited. This trend has made it challenging to capture the patterns of centralized model training and services from past end-devices. To address this problem, federal Learning (FL) is an attractive solution to collaboratively learn across various devices without sharing private data. In particular, FL is a decentralized collaborative training framework that delivers cryptographic models or gradients with the help of federal servers to collaboratively train client models. In the FL process, the model training work based on the private data is unloaded to the client side to be executed locally, and is not uploaded to the centralized training of the federal server, so that the safety of the private data is protected.
However, applying federal learning in a real scenario requires a data environment that faces spatio-temporal heterogeneity. On one hand, federal learning faces spatial heterogeneity challenges, i.e., the client's private data may not satisfy independent co-distributed assumptions. For example, the disease distribution detected by the characteristics of the cases collected in hospitals is different, and in particular, the disease distribution of patients in orthopedic hospitals and infectious disease hospitals is obviously different. Applying federal learning on such data can result in the global model update direction not being consistent with the true data distribution, resulting in slow model convergence or poor performance, i.e., spatial heterogeneity of the data. On the other hand, federal learning faces the time heterogeneity challenge, the distribution of client private data can change with time, and the distribution of global data is unstable from the perspective of the federal system. For example, the rate of allergic diseases detected in hospitals is regularly increased at 4-5 months per year due to time-related factors such as season and epidemic. I.e., data retention time heterogeneity. The two phenomena are simultaneously common in the application scene of federal learning, and in the face of spatio-temporal heterogeneous data distribution, the federal server needs to communicate with the client side at high frequency so as to keep the consistency of target distribution and real distribution of the federal global model.
The structure of the model is increased more and more by the aid of the requirements of the depth model on multi-mode information processing and the requirements on model characterization and reasoning capability while performance of the depth model is continuously improved. For example, the number of parameters of the VGG deep network commonly used for the medical image recognition task in the intelligent medical scene reaches hundreds of millions. The amount of parameters for the GPT-3 depth model, which is superior in performance and is commonly used for medical assistant tasks in intelligent medical scenarios, is even reaching the staggering billions. Frequent large-model communication can cause heavy communication cost and power consumption of mobile intelligent devices, and the problem seriously hinders the application of federal learning. Furthermore, the contradiction promotes the challenge of low-cost client-side collaborative training on the premise that the private data of the client-side is not exposed in the spatial-temporal data heterogeneous scene.
Aiming at the light weight challenge of the federal model, the existing work is respectively developed from the two aspects of reducing the number of federal communication and reducing the amount of federal communication. In the aspect of reducing the number of federal communications, the Fed average algorithm controls the communications cost of the client by adjusting the proportion of the client randomly selected in each round of federal learning. However, this approach lacks differentiation considerations for client value. The FedPNS method performs probability-based communication control on the client from the perspective of gradient consistency of the client and a federal global model, measures the importance of the client by using the similarity of a client gradient updating direction and a federal global updating gradient, and allocates the federal communication probability based on the importance measurement. In addition to gradient-based methods, methods such as fedmcs use a multi-standard client selection mechanism based on client performance to ramp communication demand towards clients with relatively abundant resources, thereby balancing the resource load of clients and reducing the communication efficiency of federal learning by reducing the communication demand of weak resource clients. On the other hand, in terms of reducing federal communication traffic, the existing work has proposed various small-model communication methods from the perspective of compression of communication model quantities. The Hermes method provides that in a federated system with inconsistent data distribution of clients, a hyper-network is generated in a federated global model for learning global knowledge, and a neural network pruning method is used for selecting private knowledge of sub-network fitting clients with fewer parameters at each client. In the federal communication process, the client and the server only interact with the intersection part of the models, so that the communication parameter quantity of a single federal process is reduced.
However, existing work mostly assumes that the distribution of clients is stable over time. Therefore, when the data distribution changes with time, the existing method can only improve the communication probability of all clients without difference, so that the federal global model can sense the data distribution change as soon as possible. This clearly further exacerbates the communication burden of the federal system.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a lightweight federated learning method and system facing to a space-time data heterogeneous scene, which can distribute different communication probabilities to clients according to the change condition of the data distribution of the clients, so that the clients which have large data distribution change degree and are not perceived by a federated global model participate in federated communication with higher probability instead of equally increasing the communication traffic of all the clients. By inclining the communication resources, the global data distribution change can be sensed by less communication traffic, so that the communication efficiency of federal learning under the space-time heterogeneous scene is improved.
In order to achieve the above purpose, the invention provides the following technical scheme:
on one hand, the invention provides a lightweight federal learning method oriented to space-time data heterogeneous scenes, which comprises the following steps:
s1, local updating of a client: after receiving the latest federated global model, updating the federated global model into a localization model which is more consistent with local private data distribution by using local private data of the client;
s2, evaluating the importance of the client: measuring the importance of the client participating in the federal communication on the global model of the federation by using the distribution change degree of private data and the consistency of the gradient information of the client model and the gradient information of the global model;
s3, controlling the communication probability of the client: dynamically adjusting the communication probability of the client according to the importance of the client obtained in the step S2, judging whether to upload according to the probability, if so, entering the step S4, and if not, issuing the federal global model to all clients;
s4, federal service end aggregation: updating the federal global model by using the gradient information uploaded by the clients participating in the federal update in the current round, issuing the updated federal global model to all the clients, and entering the next round of federal process until the preset federal turn is reached.
Further, the specific process of step S1 is: local to the client, in the federal global model ω t On the basis, by a gradient descent iterative algorithm, using a private data set of the client i in t turns
Continuing training to obtain a local model
The optimization objective function of the client is
Wherein n is
And f is a loss function of the client i. Cross-entropy loss is common in classification tasks.
Further, the specific process of step S2 is:
s21, the model obtained after the local update of the client i in the t round is
First, a model is calculated
Top-1 accuracy on recent data fragmentation
Second calculation model
Accuracy on the last n rounds of data slicing
Finally, calculating the absolute value of the difference of the accuracy rates of the data at two different time ends on the same model as
Reflect the absolute distribution change of the local data of the client, record as
S22, caching the local model uploaded to the federal server last time by the client
Compare it with local model of the current round
Obtaining gradient information of the client end which is not uploaded
Comparing the received federal global model omega which is calculated by the client after last uploading s With the latest federal global model omega t To obtain the perceived gradient information of the global model of the federation
By calculating cosine similarity of gradient information of the client and gradient information of the global federated model
Measure the relative difference between the gradient to be uploaded by the client and the perceived gradient of the global model of the federation, and record the difference as
S23, combining the absolute change AC and the relative change RC in a multiplication mode, and taking the combination as the importance measurement of the client i in the round t:
further, the specific process of step S3 is:
s31, inputting the importance metric of the client into a communication probability mapping module, and updating the communication probability of the client i into
The tau control federal system is a communication resource distributed by time change of data distribution, the sensitivity degree of the tau control system to distribution change is reflected, and larger tau can enable the federal system to adapt to the distribution change of a client more quickly, but can also generate additional higher communication overhead; p is floor Controlling the basic communication probability of the client; therefore, the uplink probability is higher when the importance of the client S is higher, and the uplink probability is lower when the importance of the client S is lower.
And S32, the communication probability control module generates a client-side unique heat vector to be uploaded, namely an OneHot vector according to the probability in each round of federal process, wherein one dimension value is 1, and the other dimension values are 0, and the corresponding client side is required to upload the gradient information of the local model.
Further, the specific process of step S4 is:
s41, uploading the gradient accumulated gradient from the S moment to the t moment in the t round by the selected client i
S42, the client set uploaded in the current round is C t Historical federal integrityThe local model is omega t Updating the accumulated gradient information uploaded to the client to a federal global model by a learning rate eta
And S43, issuing the updated global federated model to all the clients.
In another aspect, the invention also provides a lightweight federal learning system oriented to spatio-temporal data heterogeneous scenes, which comprises the following modules for realizing the method of any one of the above:
the client local updating module is used for updating the federal global model into a localization model which is more consistent with local private data distribution by using local private data of the client after receiving the latest federal global model;
the client importance evaluation module is used for measuring the importance of the client participating in the federal communication on the federal global model by utilizing the distribution change degree of private data and the consistency of the gradient information of the client model and the gradient information of the global model;
the client communication probability control module dynamically adjusts the client communication probability according to the client importance obtained by the client importance evaluation module, judges whether to upload according to the probability, enters the federal service end aggregation module if the client communication probability is uploaded, and issues the federal global model to all clients if the client communication probability control module does not upload the client communication probability;
and the federal server side aggregation module is used for updating the federal global model by using the gradient information uploaded by the clients participating in the federal update in the current round, issuing the updated federal global model to all the clients, and entering the next federal process until the preset federal turn is reached.
Compared with the prior art, the invention has the beneficial effects that:
according to the lightweight federal learning method and system for the space-time data heterogeneous scene, different communication probabilities can be distributed to the client according to the change situation of the data distribution of the client, so that the client which has a large data distribution change degree and is not perceived by a global federal model participates in federal communication with a higher probability, and the communication traffic of all clients is not uniformly improved. By inclining the communication resources, the global data distribution change can be sensed with less communication traffic, so that the communication efficiency of federal learning under the space-time heterogeneous scene is improved.
By using the lightweight federal learning method and system for spatio-temporal data heterogeneous scenes, which are provided by the invention, each image acquisition and analysis terminal calculates the importance of the image acquisition and analysis terminal in each federal process by using the change information of the private case data distribution and the gradient information of the private detection model of the image acquisition and analysis terminal. And dynamically adjusting the probability of each round of federal process image acquisition and analysis terminal participating in federal communication according to the important measurement result of the image acquisition and analysis terminal. The communication demand is distributed to the more important image acquisition and analysis terminal, the communication traffic of the less important image acquisition and analysis terminal is reduced, the same federal learning performance is achieved with less federal communication cost, and the federal learning communication traffic is effectively reduced.
Drawings
In order to more clearly illustrate the embodiments of the present application or technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments described in the present invention, and other drawings can be obtained by those skilled in the art according to the drawings.
Fig. 1 is a flowchart of a lightweight federated learning method for spatio-temporal data heterogeneous scenes according to an embodiment of the present invention.
Fig. 2 is a lightweight federal learning system architecture diagram oriented to spatio-temporal data heterogeneous scenes, provided by an embodiment of the present invention.
Detailed Description
For a better understanding of the present invention, the method of the present invention is described in detail below with reference to the accompanying drawings.
According to the method, under the scene of space-time data isomerism, the uplink communication probability of the client is dynamically adjusted in the federal process by using the distribution change degree of the client private data and the gradient information of the client model, and the communication overhead of the low-importance client is reduced. Therefore, the problem of communication efficiency bottleneck of the federal system in a space-time data heterogeneous scene is effectively solved, and the light-weight federal learning system is realized.
In a space-time data heterogeneous environment, the federal global model needs to continuously fit the changed data distribution, and due to data privacy protection, the fitting process of the distribution change needs to be completed through uploading of information of the federal client model. However, since the degree of change of the data distribution of each client over time is different from the timing, how to select the client participating in the communication when the global data distribution is fitted changes greatly affects the fitting effect and the required communication amount. In order to realize the fitting of data distribution change in a light weight manner, the invention provides a light weight federal learning method and a light weight federal learning system for a spatio-temporal data heterogeneous scene, which are shown in figures 1 and 2. The system comprises four main modules, namely a client local updating module, a client importance evaluation module, a federal communication probability control module and a federal server aggregation module.
The client local updating module firstly utilizes private data to update the global federated model into a private model locally at the client. Then, the invention introduces a client importance detection module facing the client distribution change degree to quantify the contribution of each client communication to the global model distribution change perception in each round of federal learning.
First, if the client local data distribution is stable, its upload communication cannot contribute to the fitting of global distribution changes. Therefore, the importance detection module reflects the distribution difference of local private incremental data and historical data in the current round by using the difference between the accuracy of the latest client local model on the incremental sample in the current round and the accuracy of the latest client local model on the historical sample, and the measurement method is called as client absolute distribution change degree measurement.
Secondly, since federated learning is participated by multiple clients, if the local distribution change information of one client is already perceived by the federated global model through the communication process with other clients, the importance of the client communication to the distribution change perception is lower compared with the client whose distribution change is not perceived by the federated global model. Therefore, cosine similarity of gradient information which is not uploaded by the client and gradient information of a time window corresponding to the global model is further introduced, and the relative change degree of distribution change carried by incremental data of the client and perceived distribution of the global model is reflected. If the change of the model of the client is in the same direction as the change of the global federated model, namely the similarity is high, the part of the incremental information of the client which is not sensed by the global model is less, and the corresponding importance is low. Finally, the absolute importance and the relative importance are combined in a multiplication mode to be used as weights, the degree of the distribution change of the client private data and the similarity degree of the client private data change and the global model perceived change are comprehensively considered, and the importance of the client communication on the distribution change of the federal global model perceived data is comprehensively reflected.
On the basis of measuring the importance of the client, more federal communication requirements are naturally distributed to the client which has larger contribution to the change of the distribution of the sensing data of the global federal model, so that the communication efficiency can be improved. Therefore, after the importance of the client for each round of federal learning is updated, the client communication control module adjusts the probability of uploading the model by the client according to the importance of the client. By mapping the client importance to the federal communication probability, the high-importance client has a higher probability of uploading a model to participate in federal aggregation, and the model of the low-importance client is correspondingly reduced to upload. And finally, the federal server side aggregation module is responsible for updating the model gradient information uploaded by the client side according to the probability to a historical federal global model and sending the updated federal global model to the client side.
Example 1
And (3) task description:
the federal learning system contains 10 clients, and the process of federal learning in the experiment lasts 10 rounds. The simulation method for simulating the data space heterogeneity randomly allocates 20% of data corresponding to the categories to each client in the Federal learning initialization process, and incremental data of subsequent clients come from the categories allocated in the initialization process. The simulation method of the time-varying client private data distribution is to change the data categories of 4 clients from the 5 th round. The federation procedure for client 6 in round 5 is described in detail in the case:
step 1: client local update
Step 1.1: the client receives the global model omega of the federation issued by the server 5 . Each client of the federal system is updated to a local model by using local data
Where i is the client id belonging to [1,10 ]]。
Step 2: client importance evaluation
Step 2.1: first, absolute weight detection is performed, taking client 6 as an example, which generates local private data distribution change in round 5, using local model
Testing local private incremental datasets at round 5
0.8, and a union of the local private incremental datasets over historical 2 rounds
And
the accuracy of the data set is 0.85, and further, the absolute value abs (0.80.85) of the difference between the accuracy of the two data sets is used to reflect that the difference between the data set of the current turn and the data set of the historical 2 turns is 0.05, that is, the absolute change metric value of the local data of the client 6 in the 5 th turn is 0.05.
Step 2.2: calculating the relative distribution change degree of the client 6, wherein the client model 6 uploads the local model in the 3 rd round before the 5 th round, so that the cumulative gradient from the 3 rd round to the 5 th round of the client is calculated as
Record as
Similarly, a federated global model ω for calculating the last upload time point of the client 5 And omega 3 Cumulative gradient ω of 5 -ω 3 Record as
Further, calculate
And with
Has a cosine similarity of 0.4 and a relative importance measure of 1-0.4=0.6.
Step 2.3: the importance to client 6 in round 5 is the product of its relative and absolute importance, i.e. the importance
And step 3: client communication probability control
Step 3.1: updating the communication probability of each client j into
Where the hyperparameter tau is set to 2,P floor The setting was 5%. The probability of communication of client 6 in round 5 is 6%.
Step 3.2: and (3) the client 6 carries out random number of [0,1], if the random value is less than 6%, the gradient information of the local model is uploaded to the federal server, and if not, the client does not participate in the uploading in the current round.
And 4, step 4: federal service side aggregation
4.1 the federal server updates the gradient information of the clients uploaded in the current round to a federal global model, and assumes that the set C of the clients uploaded in the current round belongs to the [6,7,9 ] E]Recording client side [6,7,9 ]]Respectively of ascending gradient
Then a new round of the federal global model
Where η is the updated learning rate of the federal global model and is set to 0.01 in the experiment.
4.2 Federal Server merging Federal Global model ω 6 And (5) issuing the data to all the clients, and entering the next round of federal process.
Experiments prove that in a scene that the distribution of 20% of clients changes along with time, the system compares Fed-average and FedPNS baseline methods, and the like, and can achieve the same performance as that of Fed-average and FedPNS methods in which 40% of clients participate in federal upload on average under the condition of communication cost uploaded by 30% of clients on average, namely, the uplink communication overhead is reduced by 10% under the condition of the same performance. The federal communication efficiency is effectively improved, and light-weight federal learning on space-time heterogeneous scenes is achieved.
In summary, the invention provides a lightweight federal learning system and a method facing to a space-time data heterogeneous scene, in the scene of the internet of things such as intelligent medical treatment and the like, the importance of a client is measured by using private data distribution change information of the client, the communication probability of the client is dynamically adjusted according to the importance in the federal process, the communication overhead of the federal system is reduced, and each image acquisition and analysis terminal calculates the importance of the image acquisition and analysis terminal in each federal process by using the change information of the private case data distribution of the image acquisition and analysis terminal and the gradient information of a private detection model of the image acquisition and analysis terminal. And dynamically adjusting the probability of each round of federal process image acquisition and analysis terminal participating in federal communication according to the important measurement result of the image acquisition and analysis terminal. The communication demand is distributed to the more important image acquisition and analysis terminal, the communication traffic of the less important image acquisition and analysis terminal is reduced, the same federal learning performance is achieved with less federal communication cost, and the federal learning communication traffic is effectively reduced.
The above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: it is to be understood that modifications may be made to the technical solutions described in the foregoing embodiments, or equivalents may be substituted for some of the technical features thereof, but such modifications or substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (6)
1. A lightweight federated learning method for space-time data heterogeneous scenes is characterized by comprising the following steps:
s1, local updating of a client: after receiving the latest federated global model, updating the federated global model into a localization model which is more consistent with local private data distribution by using local private data of the client;
s2, client importance evaluation: measuring the importance of the client participating in the federal communication on the global model of the federation by using the distribution change degree of private data and the consistency of the gradient information of the client model and the gradient information of the global model;
s3, controlling the communication probability of the client: dynamically adjusting the communication probability of the client according to the importance of the client obtained in the step S2, judging whether to upload according to the probability, if so, entering the step S4, and if not, issuing the federal global model to all the clients;
s4, federal service side aggregation: updating the global federal model by using the gradient information uploaded by the clients participating in the federal update in the current round, issuing the updated global federal model to all the clients, and entering the next federal process until the preset federal turn is reached.
2. The lightweight federal learning method for spatio-temporal data heterogeneous scenes as claimed in claim 1, wherein the specific process of the step S1 is as follows: local to the client, in the federal global model ω t On the basis, by a gradient descent iterative algorithm, using a private data set of the client i in t turns
Continue training to obtain the localModel (model)
The optimization objective function of the client is
Wherein n is
And f is a loss function of the client i.
3. The lightweight federal learning method for spatio-temporal data heterogeneous scenes as claimed in claim 1, wherein the specific process of step S2 is as follows:
s21, the model obtained after the local update of the t round of the client i is
First, a model is calculated
Top-1 accuracy on recent data fragmentation
Second calculation model
Accuracy on the last n rounds of data slicing
Finally, calculating the absolute value of the difference of the accuracy rates of the data at two different time ends on the same model as
Reflect the absolute distribution change of the local data of the client, and record it as
S22, caching the local model uploaded to the federal server last time by the client
Compare it with local model of the current round
Obtaining gradient information of the client end which is not uploaded
Comparing the received federal global model omega which is calculated by the client after last uploading s With the latest federal global model omega t Obtaining the perceptual gradient information of the global model of the federation
By calculating the cosine similarity of the gradient information of the client and the gradient information of the global federated model
The relative difference between the gradient to be uploaded at the client and the perceived gradient of the global federal model is measured and recorded as
S23, combining the absolute change AC and the relative change RC in a multiplication mode, and taking the combination as the importance measurement of the client i in the round t:
4. the lightweight federal learning method for spatio-temporal data heterogeneous scenes as claimed in claim 1, wherein the specific process of step S3 is:
s31, customerThe importance metric of the client is input into a communication probability mapping module, and the communication probability of the client i is updated to
The tau control federal system distributes communication resources for the time change of data distribution, and reflects the sensitivity degree of the distribution change; p is floor The basic communication probability of the client is controlled, the uplink probability of the high-importance client is dynamically increased, and the uplink probability of the low-importance client is reduced;
and S32, the communication probability control module generates a client-side unique heat vector to be uploaded according to the probability in each round of federal process, wherein one dimension value is 1, and the other dimension values are 0, and the corresponding client side is required to upload the gradient information of the local model.
5. The lightweight federal learning method for spatio-temporal data heterogeneous scenes as claimed in claim 1, wherein the specific process of step S4 is:
s41, uploading the gradient accumulated gradient from the S moment to the t moment in the t round by the selected client i
S42, the client set uploaded in the current round is C t The historical federal global model is ω t Updating the accumulated gradient information uploaded to the client to a federal global model by a learning rate eta
And S43, issuing the updated global federated model to all clients.
6. A lightweight federated learning system oriented to spatio-temporal data heterogeneous scenarios, comprising the following modules to implement the method of any one of claims 1-5:
the client local updating module is used for updating the federal global model into a localization model which is more in line with the distribution of local private data by using local private data of the client after receiving the latest federal global model;
the client importance evaluation module is used for measuring the importance of the client participating in the federal communication on the federal global model by utilizing the distribution change degree of private data and the consistency of the gradient information of the client model and the gradient information of the global model;
the client communication probability control module dynamically adjusts the client communication probability according to the client importance obtained by the client importance evaluation module, judges whether to upload according to the probability, enters the federal service end aggregation module if the client communication probability is uploaded, and issues the federal global model to all clients if the client communication probability control module does not upload the client communication probability;
and the federal server side aggregation module is used for updating the federal global model by using the gradient information uploaded by the clients participating in the federal update in the current round, issuing the updated federal global model to all the clients and entering the next federal process until the preset federal turn is reached.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211649919.4A CN115775025A (en) | 2022-12-21 | 2022-12-21 | Lightweight federated learning method and system for space-time data heterogeneous scene |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211649919.4A CN115775025A (en) | 2022-12-21 | 2022-12-21 | Lightweight federated learning method and system for space-time data heterogeneous scene |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115775025A true CN115775025A (en) | 2023-03-10 |
Family
ID=85393253
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211649919.4A Pending CN115775025A (en) | 2022-12-21 | 2022-12-21 | Lightweight federated learning method and system for space-time data heterogeneous scene |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115775025A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115018085A (en) * | 2022-05-23 | 2022-09-06 | 郑州大学 | Data heterogeneity-oriented federated learning participation equipment selection method |
-
2022
- 2022-12-21 CN CN202211649919.4A patent/CN115775025A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115018085A (en) * | 2022-05-23 | 2022-09-06 | 郑州大学 | Data heterogeneity-oriented federated learning participation equipment selection method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113112027B (en) | Federal learning method based on dynamic adjustment of model aggregation weight | |
| WO2021026944A1 (en) | Adaptive transmission method for industrial wireless streaming media employing particle swarm and neural network | |
| WO2017012321A1 (en) | Display state adjustment method, display state adjustment device and display device | |
| CN113191503B (en) | Decentralized distributed learning method and system for non-shared data | |
| Kasturi et al. | Fusion learning: A one shot federated learning | |
| CN114912705A (en) | Optimization method for heterogeneous model fusion in federated learning | |
| CN117523291A (en) | Image classification method based on federal knowledge distillation and ensemble learning | |
| CN111445010B (en) | Distribution network voltage trend early warning method based on evidence theory fusion quantum network | |
| CN115587633A (en) | Personalized federal learning method based on parameter layering | |
| CN115775025A (en) | Lightweight federated learning method and system for space-time data heterogeneous scene | |
| CN113206887A (en) | Method for accelerating federal learning aiming at data and equipment isomerism under edge calculation | |
| CN114398634A (en) | Federal learning participant weight calculation method based on information entropy | |
| Pan et al. | Multi‐source information art painting fusion interactive 3d dynamic scene virtual reality technology application research | |
| CN109634211A (en) | AC servo identification Method and control system based on frequency data | |
| CN115829063A (en) | Wireless positioning differential privacy federation learning method based on dynamic privacy budget | |
| CN108765540A (en) | A kind of heavy illumination method based on image and integrated study | |
| CN117994635B (en) | Federal element learning image recognition method and system with enhanced noise robustness | |
| CN117131918A (en) | Federal learning model layer clipping method and system based on differential privacy | |
| Pang et al. | Federated learning for crowd counting in smart surveillance systems | |
| Castellon et al. | Federated learning with incremental clustering for heterogeneous data | |
| CN113095513A (en) | Double-layer fair federal learning method, device and storage medium | |
| CN117744758A (en) | Federal learning optimization method based on mobile equipment | |
| CN114912627A (en) | Recommendation model training method, system, computer device and storage medium | |
| CN114909707A (en) | Heat supply secondary network regulation and control method based on intelligent balancing device and reinforcement learning | |
| CN116050557A (en) | Power load prediction method, device, computer equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |