CN115775025A - Lightweight federated learning method and system for space-time data heterogeneous scene - Google Patents

Lightweight federated learning method and system for space-time data heterogeneous scene Download PDF

Info

Publication number
CN115775025A
CN115775025A CN202211649919.4A CN202211649919A CN115775025A CN 115775025 A CN115775025 A CN 115775025A CN 202211649919 A CN202211649919 A CN 202211649919A CN 115775025 A CN115775025 A CN 115775025A
Authority
CN
China
Prior art keywords
client
model
federal
communication
federated
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211649919.4A
Other languages
Chinese (zh)
Inventor
张宇超
李嘉晨
王文东
龚向阳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing University of Posts and Telecommunications
Original Assignee
Beijing University of Posts and Telecommunications
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing University of Posts and Telecommunications filed Critical Beijing University of Posts and Telecommunications
Priority to CN202211649919.4A priority Critical patent/CN115775025A/en
Publication of CN115775025A publication Critical patent/CN115775025A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Computer And Data Communications (AREA)

Abstract

本发明公开了一种面向时空数据异构场景的轻量化联邦学习方法及系统,能根据客户端数据分布的变化情况给客户端分配不同的通信概率,从而让数据分布变化程度大,且未被联邦全局模型感知的客户端,以更高的概率参与联邦通信,而不是均等的提升所有客户端的通信量。通过将通信资源倾斜,可以用更少的通信量感知全局数据分布变化,提升在时空异构场景下联邦学习的通信效率。通过使用本发明提出的方法及系统,各图像采集分析终端利用其私有病例数据分布的变化信息以及图像采集分析终端私有检测模型的梯度信息,动态的调整每一轮联邦过程图像采集分析终端参与联邦通信的概率,实现以更少的联邦通信成本达到相同联邦学习性能,有效的降低联邦学习通信量。

Figure 202211649919

The invention discloses a lightweight federated learning method and system oriented to spatio-temporal data heterogeneous scenarios, which can assign different communication probabilities to clients according to changes in client data distribution, so that the degree of data distribution changes greatly and is not affected by Clients perceived by the federated global model participate in federated communication with a higher probability, instead of increasing the communication traffic of all clients equally. By tilting communication resources, it is possible to sense global data distribution changes with less communication traffic, and improve the communication efficiency of federated learning in spatiotemporal heterogeneous scenarios. By using the method and system proposed by the present invention, each image collection and analysis terminal uses the change information of its private case data distribution and the gradient information of the private detection model of the image collection and analysis terminal to dynamically adjust the participation of image collection and analysis terminals in each round of federation process. The probability of communication can achieve the same federated learning performance with less federated communication cost, effectively reducing the federated learning traffic.

Figure 202211649919

Description

一种面向时空数据异构场景的轻量化联邦学习方法及系统A lightweight federated learning method and system for spatiotemporal data heterogeneous scenarios

技术领域technical field

本发明涉及联邦学习技术领域,尤其涉及一种面向时空数据异构场景的轻量化联邦学习方法及系统。The invention relates to the technical field of federated learning, in particular to a lightweight federated learning method and system for heterogeneous spatiotemporal data scenarios.

背景技术Background technique

深度学习技术近年来显著的提升了计算机视觉,自然语言处理等任务的性能。通过在海量数据上训练百万级参数的深度神经网络(DNN),深度学习模型的性能逐渐超越传统机器学习成为人工智能技术工业落地的主流。传统深度学习因其对海量训练数据的依赖,通常采用在边缘设备采集数据,上传云端服务器进行模型训练与推理服务的中心化模式。In recent years, deep learning technology has significantly improved the performance of tasks such as computer vision and natural language processing. By training a deep neural network (DNN) with millions of parameters on massive data, the performance of deep learning models has gradually surpassed traditional machine learning and has become the mainstream of the artificial intelligence technology industry. Due to its dependence on massive training data, traditional deep learning usually adopts a centralized model of collecting data on edge devices and uploading it to cloud servers for model training and inference services.

然而近年来,随着数据安全和个人隐私保护的法案逐步出台,将从手机、平板等用户端设备采集的私有数据上传到中央服务器这一过程面临了诸多限制。这一趋势使得以往的端设备采集中心化模型训练与服务的模式变得充满挑战。为解决此问题,联邦学习(FL)是一个有吸引力的解决方案,可以在不共享私人数据的情况下跨各种设备进行协作学习。具体来说,FL是一个去中心化的协同训练框架,在联邦服务器的帮助下传递加密模型或梯度来协同训练各客户端模型。在FL过程中,基于私有数据的模型训练工作被卸载到客户端本地执行,而非上传到联邦服务器集中式训练,由此保护了私有数据的安全性。However, in recent years, with the gradual promulgation of data security and personal privacy protection bills, the process of uploading private data collected from mobile phones, tablets and other client devices to the central server faces many restrictions. This trend makes the previous mode of centralized model training and service of end device collection full of challenges. To address this problem, federated learning (FL) is an attractive solution that enables collaborative learning across various devices without sharing private data. Specifically, FL is a decentralized collaborative training framework that transfers encrypted models or gradients with the help of federated servers to collaboratively train each client model. In the FL process, the model training work based on private data is offloaded to the client for local execution, rather than uploaded to the federated server for centralized training, thereby protecting the security of private data.

然而,在实际场景中应用联邦学习需要面对时空异构的数据环境。一方面,联邦学习面临空间异构性挑战,即客户端的私有数据可能不满足独立同分布假设。例如,各医院收治病例特点导致的检出病因分布差异,具体的,骨科医院与传染病医院的患者病因分布会存在明显差异。在这样的数据上应用联邦学习会导致全局模型更新方向与真实数据分布不一致,导致模型收敛缓慢或性能不佳,即数据存在空间异构性。另一方面,联邦学习面临时间异构性挑战,客户端私有数据的分布会随时间变化,从联邦系统的角度看,全局数据的分布是不稳定的。例如,由于季节与流行病等时间相关因素,导致各医院的检出过敏类疾疾病比例在每年4-5月规律性升高。即数据存时间异构性。以上两种现象同时常见于联邦学习的应用场景中,面对时空异构的数据分布,联邦服务器需要与客户端高频的通信以保持联邦全局模型的目标分布与真实分布的一致性。However, the application of federated learning in practical scenarios needs to face the spatiotemporal heterogeneous data environment. On the one hand, federated learning faces the challenge of spatial heterogeneity, that is, the client's private data may not satisfy the independent and identical distribution assumption. For example, the distribution of detected causes is different due to the characteristics of cases admitted to each hospital. Specifically, the distribution of causes of patients in orthopedic hospitals and infectious disease hospitals will be significantly different. Applying federated learning on such data will cause the global model update direction to be inconsistent with the real data distribution, resulting in slow model convergence or poor performance, that is, spatial heterogeneity in the data. On the other hand, federated learning faces the challenge of time heterogeneity. The distribution of client private data will change over time. From the perspective of the federated system, the distribution of global data is unstable. For example, due to time-related factors such as seasons and epidemics, the proportion of allergic diseases detected in each hospital regularly increases from April to May every year. That is, data storage time heterogeneity. The above two phenomena are common in the application scenarios of federated learning. In the face of heterogeneous data distribution in time and space, the federated server needs to communicate frequently with the client to maintain the consistency between the target distribution of the federated global model and the real distribution.

深度模型在性能不断提升的同时,对多模态信息处理的诉求以及模型表征与推理能力的诉求,都使得模型的结构也变得越来越大。如智能医疗场景中医学图像识别任务常用的VGG深度网络的参数量已达亿级。性能出众且常用于智能医疗场景场景中医疗助手任务的GPT-3深度模型的参数量甚至达到惊人的千亿级。频繁的大模型通信会导致沉重的通信成本和移动智能设备功耗,这一问题严重阻碍了联邦学习的应用。进而,上述矛盾催生了在时空数据异构场景下客户端私有数据不暴露前提下,低成本客户端协同训练的挑战。While the performance of the deep model continues to improve, the demand for multi-modal information processing and the demand for model representation and reasoning capabilities make the structure of the model larger and larger. For example, the VGG deep network commonly used in medical image recognition tasks in intelligent medical scenarios has reached 100 million parameters. The GPT-3 deep model, which has outstanding performance and is often used for medical assistant tasks in intelligent medical scenarios, even has an astonishing 100 billion parameters. Frequent communication of large models will lead to heavy communication costs and power consumption of mobile smart devices, which seriously hinders the application of federated learning. Furthermore, the above contradictions give rise to the challenge of low-cost client collaborative training under the premise of not exposing client private data in the spatio-temporal data heterogeneous scenario.

针对联邦模型的轻量化挑战,现有工作分别从减少联邦通信次数与减少联邦通信量两个角度展开。其中,在减少联邦通信次数方面,Fed average算法通过调整每一轮联邦学习中随机选择的客户端比例,控制客户端的通信成本。然而该方法缺少对客户端价值的差异化考虑。FedPNS方法从客户端与联邦全局模型的梯度一致性角度对客户端做基于概率的通信控制,该方法使用客户端梯度更新方向与联邦全局更新梯度的相似性衡量客户端的重要性,并基于此重要性度量分配联邦的通信概率。除了基于梯度的方法,FedMCCS等方法使用基于客户端性能的多标准客户端选择机制,将通信需求向资源相对充裕的客户端倾斜,从而均衡客户端的资源负载并通过减少弱资源客户端的通信需求,降低联邦学习的通信效率。另一方面,在减少联邦通信量方面,现有工作从通信模型量的压缩角度出发,提出了多种小模型通信方法。Hermes方法提出在客户端数据分布不一致的联邦系统中,在联邦全局模型生成超网络用于学习全局知识,并在每个客户端利用神经网络剪枝方法选择具有更少参数量的子网络拟合客户端的私有知识。在联邦通信过程中,客户端与服务端仅交互模型的交集部分,从而降低单次联邦过程的通信参数量。In response to the lightweight challenge of the federated model, existing work is carried out from two perspectives: reducing the number of federated communications and reducing the amount of federated communications. Among them, in terms of reducing the number of federated communications, the Fed average algorithm controls the communication cost of clients by adjusting the proportion of randomly selected clients in each round of federated learning. However, this method lacks the consideration of the differentiation of client value. The FedPNS method performs probability-based communication control on the client from the perspective of the gradient consistency between the client and the federation global model. This method uses the similarity between the client gradient update direction and the federation global update gradient to measure the importance of the client, and based on this important The property measures the communication probability of the distribution federation. In addition to the gradient-based method, methods such as FedMCCS use a multi-standard client selection mechanism based on client performance to tilt the communication demand to clients with relatively abundant resources, thereby balancing the resource load of the client and reducing the communication demand of weak resource clients. Reduce the communication efficiency of federated learning. On the other hand, in terms of reducing the amount of federated communication, existing work proposes a variety of small-model communication methods from the perspective of compressing the amount of communication models. The Hermes method proposes that in a federated system with inconsistent client data distribution, the federated global model generates a supernetwork for learning global knowledge, and uses the neural network pruning method to select a subnetwork with fewer parameters for each client. Client's private knowledge. In the process of federated communication, the client and server only interact with the intersection part of the model, thereby reducing the amount of communication parameters in a single federated process.

然而,现有工作大多假设客户端的分布随时间稳定。因此当数据分布随时间发生变化时,现有方法只能无差异的提升所有客户端的通信概率使联邦全局模型尽快感知数据分布变化。这无疑进一步的恶化了联邦系统的通信负担。However, most of the existing works assume that the distribution of clients is stable over time. Therefore, when the data distribution changes over time, the existing methods can only increase the communication probability of all clients indiscriminately so that the federated global model can perceive the change of data distribution as soon as possible. This undoubtedly further exacerbated the communication burden on the federal system.

发明内容Contents of the invention

本发明针对现有技术的不足,提出一种面向时空数据异构场景的轻量化联邦学习方法及系统,能根据客户端数据分布的变化情况给客户端分配不同的通信概率,从而让数据分布变化程度大,且未被联邦全局模型感知的客户端,以更高的概率参与联邦通信,而不是均等的提升所有客户端的通信量。通过将通信资源倾斜,可以用更少的通信量感知全局数据分布变化,从而提升在时空异构场景下,联邦学习的通信效率。Aiming at the deficiencies of the prior art, the present invention proposes a lightweight federated learning method and system for spatio-temporal data heterogeneous scenarios, which can assign different communication probabilities to clients according to changes in client data distribution, thereby allowing data distribution to change Clients with a large degree and not perceived by the federated global model participate in federated communication with a higher probability, instead of increasing the communication traffic of all clients equally. By slanting communication resources, it is possible to perceive global data distribution changes with less communication traffic, thereby improving the communication efficiency of federated learning in spatiotemporal heterogeneous scenarios.

为了实现上述目的,本发明提供如下技术方案:In order to achieve the above object, the present invention provides the following technical solutions:

一方面,本发明提供一种面向时空数据异构场景的轻量化联邦学习方法,包括以下步骤:On the one hand, the present invention provides a lightweight federated learning method for heterogeneous spatiotemporal data scenarios, including the following steps:

S1、客户端本地更新:接收到最新的联邦全局模型后,利用客户端本地的私有数据,将联邦全局模型更新为更符合本地私有数据分布的本地化模型;S1. Client local update: After receiving the latest federated global model, use the local private data of the client to update the federated global model to a localized model that is more in line with the distribution of local private data;

S2、客户端重要性评价:利用私有数据的分布变化程度以及客户端模型的梯度信息与全局模型梯度信息的一致性,衡量客户端参与联邦通信对联邦全局模型的重要性;S2. Client importance evaluation: use the degree of change in the distribution of private data and the consistency between the gradient information of the client model and the gradient information of the global model to measure the importance of the client's participation in federated communication to the federated global model;

S3、客户端通信概率控制:根据S2步骤得到的客户端重要性,动态调整客户端通信概率,依概率判断是否上传,若是则进入步骤S4,若否则将联邦全局模型下发到所有客户端;S3. Client communication probability control: according to the client importance obtained in step S2, dynamically adjust the client communication probability, judge whether to upload according to the probability, if so, enter step S4, if not, send the federation global model to all clients;

S4、联邦服务端聚合:利用本轮参与联邦更新的客户端所上传的梯度信息更新联邦全局模型,并将更新后的联邦全局模型下发到所有客户端,并进入下一轮联邦过程,直到达到预设联邦轮次。S4. Federation server aggregation: use the gradient information uploaded by the clients participating in the federation update in this round to update the federation global model, and deliver the updated federation global model to all clients, and enter the next round of federation process until Reach the preset federation round.

进一步地,步骤S1的具体过程为:在客户端本地,在联邦全局模型为ωt的基础上通过梯度下降迭代算法,使用客户端i在t轮次的私有数据集

Figure BDA0004009983930000031
继续训练,得到本地模型
Figure BDA0004009983930000041
户端的优化目标函数为
Figure BDA0004009983930000042
其中,n为
Figure BDA0004009983930000043
中样本量,f为客户端i的损失函数。在分类任务中通常为交叉熵损失。Further, the specific process of step S1 is: locally on the client, on the basis of the federated global model ω t , use the private data set of client i in round t through the gradient descent iterative algorithm
Figure BDA0004009983930000031
Continue training to get the local model
Figure BDA0004009983930000041
The optimization objective function of the client is
Figure BDA0004009983930000042
Among them, n is
Figure BDA0004009983930000043
Medium sample size, f is the loss function of client i. In classification tasks it is usually cross-entropy loss.

进一步地,步骤S2的具体过程为:Further, the specific process of step S2 is:

S21、客户端i在第t轮本地更新后得到的模型为

Figure BDA0004009983930000044
首先计算模型
Figure BDA0004009983930000045
在最新数据分篇上的Top-1准确率
Figure BDA0004009983930000046
其次计算模型
Figure BDA0004009983930000047
在最近n轮数据分片上的准确率
Figure BDA0004009983930000048
最后计算两个不同时间端数据在同模型上的准确率差异的绝对值为
Figure BDA0004009983930000049
反映客户端本地数据绝对分布变化,记做
Figure BDA00040099839300000410
S21. The model obtained by client i after the tth round of local update is
Figure BDA0004009983930000044
First calculate the model
Figure BDA0004009983930000045
Top-1 accuracy rate on the latest data subsection
Figure BDA0004009983930000046
Second calculation model
Figure BDA0004009983930000047
Accuracy on the last n rounds of data sharding
Figure BDA0004009983930000048
Finally, calculate the absolute value of the accuracy difference between two different time-end data on the same model
Figure BDA0004009983930000049
Reflect the absolute distribution change of the client's local data, record as
Figure BDA00040099839300000410

S22、客户端缓存最近一次上传给联邦服务器的本地模型

Figure BDA00040099839300000411
对比其与本轮的本地模型
Figure BDA00040099839300000412
的差异,得到未上传客户端梯度信息
Figure BDA00040099839300000413
对比客户端计算最近一次上传后接收的联邦全局模型ωs与最新联邦全局模型ωt的差异,得到联邦全局模型已感知梯度信息
Figure BDA00040099839300000414
通过计算客户端梯度信息与联邦全局模型梯度信息的余弦相似度
Figure BDA00040099839300000415
衡量客户端待上传梯度与联邦全局模型已感知梯度的相对差异,记做
Figure BDA00040099839300000416
S22. The client caches the latest local model uploaded to the federated server
Figure BDA00040099839300000411
Compare it with the local model for this round
Figure BDA00040099839300000412
The difference, get the gradient information of the client that has not been uploaded
Figure BDA00040099839300000413
Compare the difference between the federated global model ω s received by the client after the last upload and the latest federated global model ω t , and obtain the perceived gradient information of the federated global model
Figure BDA00040099839300000414
By calculating the cosine similarity between the gradient information of the client and the gradient information of the federation global model
Figure BDA00040099839300000415
Measure the relative difference between the gradient to be uploaded by the client and the perceived gradient of the federated global model, and record as
Figure BDA00040099839300000416

S23、将绝对变化AC与相对变化RC以乘法形式结合,作为客户端i在t轮的重要性度量:

Figure BDA00040099839300000417
S23. Combining the absolute change AC and the relative change RC in the form of multiplication as the importance measure of client i in round t:
Figure BDA00040099839300000417

进一步地,步骤S3的具体过程为:Further, the specific process of step S3 is:

S31、将客户端重要性度量值输入通信概率映射模块,客户端i的通信概率更新为

Figure BDA00040099839300000418
其中tau控制联邦系统为数据分布的时间变化分配的通信资源,反应对分布变化的敏感程度,较大的tau将使联邦系统更快的适应客户端的分布变化,但也会产生额外的较高的通信开销;Pfloor控制客户端的基础通信概率;因此客户端S的重要性较高则上行概率较大,反之客户端重要性较低则上行概率较小。S31. Input the client importance measure value into the communication probability mapping module, and the communication probability of client i is updated as
Figure BDA00040099839300000418
Among them, tau controls the communication resources allocated by the federated system for time changes in data distribution, reflecting the sensitivity to distribution changes. A larger tau will enable the federated system to adapt to client distribution changes faster, but it will also generate additional higher Communication overhead; P floor controls the basic communication probability of the client; therefore, the higher the importance of the client S, the higher the uplink probability, and the lower the importance of the client, the lower the uplink probability.

S32、通信概率控制模块在每一轮联邦过程,依概率生成待上传客户端独热向量,即OneHot向量,其中一维值为1,其他维值为0,并要求对应客户端上传本地模型的梯度信息。S32. The communication probability control module generates a one-hot vector of the client to be uploaded according to the probability in each round of federation process, that is, a OneHot vector, in which the value of one dimension is 1, and the value of other dimensions is 0, and requires the corresponding client to upload the local model gradient information.

进一步地,步骤S4的具体过程为:Further, the specific process of step S4 is:

S41、被选中客户端i上传在第t轮自s时刻到t时刻的梯度累计梯度

Figure BDA0004009983930000051
S41. The selected client i uploads the cumulative gradient of the gradient from time s to time t in round t
Figure BDA0004009983930000051

S42、本轮上传的客户端集为Ct,历史联邦全局模型为ωt,将上传客户端的累积梯度信息以学习率η更新到联邦全局模型

Figure BDA0004009983930000052
S42. The client set uploaded in this round is C t , and the historical federated global model is ω t . Update the cumulative gradient information of uploaded clients to the federated global model at a learning rate η
Figure BDA0004009983930000052

S43、将更新后的联邦全局模型下发到所有客户端。S43. Deliver the updated federated global model to all clients.

另一方面,本发明还提供了一种面向时空数据异构场景的轻量化联邦学习系统,包括以下模块用以实现以上任一项所述的方法:On the other hand, the present invention also provides a lightweight federated learning system for spatiotemporal data heterogeneous scenarios, including the following modules to implement any of the methods described above:

客户端本地更新模块,用于接收到最新的联邦全局模型后,利用客户端本地的私有数据,将联邦全局模型更新为更符合本地私有数据分布的本地化模型;The client local update module is used to update the federated global model to a localized model that is more in line with the distribution of local private data by using the local private data of the client after receiving the latest federated global model;

客户端重要性评价模块,用于利用私有数据的分布变化程度以及客户端模型的梯度信息与全局模型梯度信息的一致性,衡量客户端参与联邦通信对联邦全局模型的重要性;The client importance evaluation module is used to measure the importance of the client's participation in federated communication to the federated global model by using the degree of distribution change of private data and the consistency between the gradient information of the client model and the gradient information of the global model;

客户端通信概率控制模块,根据客户端重要性评价模块得到的客户端重要性,动态调整客户端通信概率,依概率判断是否上传,若是则进入联邦服务端聚合模块,若否则将联邦全局模型下发到所有客户端;The client communication probability control module dynamically adjusts the client communication probability according to the client importance obtained by the client importance evaluation module, and judges whether to upload according to the probability. sent to all clients;

联邦服务端聚合模块,用于利用本轮参与联邦更新的客户端所上传的梯度信息更新联邦全局模型,并将更新后的联邦全局模型下发到所有客户端,并进入下一轮联邦过程,直到达到预设联邦轮次。The federation server aggregation module is used to update the federation global model by using the gradient information uploaded by the clients participating in the federation update in this round, and deliver the updated federated global model to all clients, and enter the next round of federation process, Until the preset federation round is reached.

与现有技术相比,本发明的有益效果为:Compared with prior art, the beneficial effect of the present invention is:

本发明提出的面向时空数据异构场景的轻量化联邦学习方法及系统,能根据客户端数据分布的变化情况给客户端分配不同的通信概率,从而让数据分布变化程度大,且未被联邦全局模型感知的客户端,以更高的概率参与联邦通信,而不是均等的提升所有客户端的通信量。通过将通信资源倾斜,可以用更少的通信量感知全局数据分布变化,从而提升在时空异构场景下,联邦学习的通信效率。The lightweight federated learning method and system for spatio-temporal data heterogeneous scenarios proposed by the present invention can assign different communication probabilities to the client according to the change of client data distribution, so that the data distribution changes greatly and is not affected by the federated global Model-aware clients participate in federated communication with a higher probability, rather than increasing the traffic of all clients equally. By slanting communication resources, it is possible to perceive global data distribution changes with less communication traffic, thereby improving the communication efficiency of federated learning in spatiotemporal heterogeneous scenarios.

通过使用本发明提出的面向时空数据异构场景的轻量化联邦学习方法及系统,各图像采集分析终端利用其私有病例数据分布的变化信息以及图像采集分析终端私有检测模型的梯度信息,在每一轮联邦过程中计算其重要性。并根据图像采集分析终端重要度量结果,动态的调整每一轮联邦过程图像采集分析终端参与联邦通信的概率。通过将通信需求分配给更重要的图像采集分析终端,降低低重要性图像采集分析终端的通信量,实现以更少的联邦通信成本达到相同联邦学习性能,有效的降低联邦学习通信量。By using the lightweight federated learning method and system oriented to heterogeneous spatiotemporal data scenarios proposed by the present invention, each image collection and analysis terminal uses the change information of its private case data distribution and the gradient information of the private detection model of the image collection and analysis terminal, in each Its importance is calculated during a round of federation. And according to the important measurement results of the image collection and analysis terminal, dynamically adjust the probability of each round of federation process image collection and analysis terminal participating in the federation communication. By allocating communication needs to more important image acquisition and analysis terminals, reducing the communication traffic of low-importance image acquisition and analysis terminals, achieving the same federated learning performance with less federated communication costs, and effectively reducing federated learning traffic.

附图说明Description of drawings

为了更清楚地说明本申请实施例或现有技术中的技术方案,下面将对实施例中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本发明中记载的一些实施例,对于本领域普通技术人员来讲,还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings that are required in the embodiments. Obviously, the accompanying drawings in the following description are only described in the present invention For some embodiments of the present invention, those skilled in the art can also obtain other drawings according to these drawings.

图1为本发明实施例提供的面向时空数据异构场景的轻量化联邦学习方法的流程图。Fig. 1 is a flow chart of a lightweight federated learning method for spatiotemporal data heterogeneous scenarios provided by an embodiment of the present invention.

图2为本发明实施例提供的面向时空数据异构场景的轻量化联邦学习系统架构图。Fig. 2 is an architecture diagram of a lightweight federated learning system oriented to spatio-temporal data heterogeneous scenarios provided by an embodiment of the present invention.

具体实施方式Detailed ways

为了更好地理解本技术方案,下面结合附图对本发明的方法做详细的说明。In order to better understand the technical solution, the method of the present invention will be described in detail below in conjunction with the accompanying drawings.

本发明在时空数据异构的场景下,利用客户端私有数据的分布变化程度与客户端模型的梯度信息,在联邦过程中动态的调整客户端的上行通信概率,减少了低重要性客户端的通信开销。从而有效解决了时空数据异构场景中联邦系统的通信效率瓶颈问题,实现了轻量化的联邦学习系统。In the scenario of spatio-temporal data heterogeneity, the present invention utilizes the distribution change degree of the client private data and the gradient information of the client model to dynamically adjust the uplink communication probability of the client during the federation process, reducing the communication overhead of low-importance clients . This effectively solves the communication efficiency bottleneck problem of the federated system in heterogeneous spatio-temporal data scenarios, and realizes a lightweight federated learning system.

在时空数据异构的环境中,联邦全局模型需要不断拟合变化的数据分布,由于数据隐私保护,分布变化的拟合过程需要通过联邦客户端模型信息的上传完成。然而,由于各客户端的数据分布随时间变化程度与时机不同,在拟合全局数据分布变化时,如何选择参与通信的客户端,会大大影响拟合效果和所需的通信量。为了轻量化的实现数据分布变化的拟合,本发明提出的面向时空数据异构场景的轻量化联邦学习方法和系统,如图1和图2所示。系统包含四个主要模块,分别是客户端本地更新模块、客户端重要性评价模块,联邦通信概率控制模块,以及联邦服务端聚合模块。In an environment with heterogeneous spatiotemporal data, the federated global model needs to continuously fit the changing data distribution. Due to data privacy protection, the fitting process of the distribution change needs to be completed by uploading the federated client model information. However, since the data distribution of each client changes with time and the timing is different, when fitting the global data distribution change, how to select the client participating in the communication will greatly affect the fitting effect and the required communication volume. In order to lightly realize the fitting of data distribution changes, the lightweight federated learning method and system for spatio-temporal data heterogeneous scenarios proposed by the present invention are shown in Fig. 1 and Fig. 2 . The system consists of four main modules, which are client local update module, client importance evaluation module, federated communication probability control module, and federated server aggregation module.

客户端本地更新模块首先在客户端本地利用私有数据更联邦全局模型为私有模型。而后本发明引入面向客户端分布变化程度的客户端重要性检测模块,来量化每一轮联邦学习中,各客户端通信对全局模型分布变化感知的贡献。The local update module of the client first uses the private data locally on the client to change the federated global model into a private model. Then, the present invention introduces a client importance detection module oriented to the change degree of client distribution to quantify the contribution of each client communication to the global model distribution change perception in each round of federated learning.

首先,如果客户端本地数据分布是稳定的,其上传通信无法为全局分布变化的拟合带来贡献。因此重要性检测模块利用最新客户端本地模型在当前轮次增量样本上的准确率与在历史样本上的准确率的差异,反应本轮本地私有增量数据和历史数据的分布差异,将该度量方法称为客户端绝对分布变化程度度量。First, if the client's local data distribution is stable, its upload communication cannot contribute to the fitting of global distribution changes. Therefore, the importance detection module uses the difference between the accuracy rate of the latest client local model on the current round of incremental samples and the accuracy rate on historical samples to reflect the distribution difference between the current round of local private incremental data and historical data. The measure is called the client absolute distribution change measure.

其次,由于联邦学习由多客户端参与,如果一个客户端本地的分布变化信息已经被联邦全局模型通过与其他客户端的通信过程感知,则该客户端通信对分布变化感知的重要性相较于分布变化未被联邦全局模型感知的客户端更低。因此,进一步引入客户端未上传的梯度信息与全局模型对应时间窗口的梯度信息的余弦相似度,反应客户端增量数据所携带的分布变化与全局模型已感知分布的相对变化程度。如果客户端的模型变化与联邦全局模型的变化同向即相似度较高,则客户端增量信息中未被全局模型感知的部分较少,对应的重要性较低。最后,通过将绝对重要性与相对重要性以乘法方式结合互为权重,综合的考虑了客户端私有数据分布变化的程度,以及客户端私有数据变化与全局模型已感知变化的相似程度,全面的反映了和客户端通信对联邦全局模型感知数据分布变化的重要性。Secondly, since federated learning involves multiple clients, if a client's local distribution change information has been perceived by the federated global model through the communication process with other clients, the importance of the client's communication for the perception of distribution changes is more important than that of distribution Clients whose changes are not perceived by the federated global model are lower. Therefore, the cosine similarity between the gradient information not uploaded by the client and the gradient information of the global model corresponding to the time window is further introduced to reflect the relative change degree of the distribution change carried by the incremental data of the client and the perceived distribution of the global model. If the client's model change is in the same direction as the federated global model's change, that is, the similarity is high, the part of the client's incremental information that is not perceived by the global model is small, and the corresponding importance is low. Finally, by combining the absolute importance and the relative importance in a multiplicative manner as weights, the degree of distribution change of the client's private data and the similarity between the change of the client's private data and the perceived change of the global model are comprehensively considered. It reflects the importance of communicating with the client to the federated global model to perceive changes in data distribution.

在度量客户端重要性的基础上,自然的将联邦通信需求更多的分配给对联邦全局模型感知数据分布变化贡献较大的客户端能提升通信效率。因此在每一轮联邦学习的客户端重要性更新后,客户端通信控制模块根据客户端的重要性,调整客户端上传模型的概率。通过将客户端重要性单调的映射为联邦通信概率,高重要性客户端将有更高的概率上传模型参与联邦聚合,并相对应的降低低重要性客户端的模型上传。最后,联邦服务端聚合模块负责将客户端依概率上传的模型梯度信息更新到历史联邦全局模型,并将更新后的联邦全局模型发给客户端。On the basis of measuring the importance of clients, it is natural to allocate more federated communication requirements to clients that contribute more to the changes in the distribution of federated global model-perceived data, which can improve communication efficiency. Therefore, after the importance of the client is updated in each round of federated learning, the client communication control module adjusts the probability of the client uploading the model according to the importance of the client. By monotonically mapping client importance to federated communication probability, high-importance clients will have a higher probability of uploading models to participate in federated aggregation, and correspondingly reduce the model uploading of low-importance clients. Finally, the federation server aggregation module is responsible for updating the model gradient information uploaded by the client according to the probability to the historical federated global model, and sends the updated federated global model to the client.

实施例1Example 1

任务描述:mission details:

联邦学习系统包含10个客户端,实验中联邦学习过程持续10个回合。模拟数据空间异构性的模拟方法为每个客户端在联邦学习初始化过程中随机分配20%的类别对应数据,后续客户端的增量数据来自于初始化过程分配类别。客户端私有数据分布随时间变化的模拟方法为从第5轮开始,令4个客户端的数据类别改变。在案例中详细描述第5轮中客户端6的联邦过程:The federated learning system contains 10 clients, and the federated learning process lasts for 10 rounds in the experiment. The simulation method of simulating the spatial heterogeneity of data randomly allocates 20% of the category-corresponding data to each client during the federated learning initialization process, and the incremental data of subsequent clients comes from the category allocation during the initialization process. The simulation method of client private data distribution changing over time is to change the data categories of 4 clients starting from the fifth round. The federation process of client 6 in round 5 is described in detail in the case:

步骤1:客户端本地更新Step 1: Client local update

步骤1.1:客户端接收由服务器下发的联邦全局模型ω5。联邦系统的各客户端利用本地数据更新为本地模型

Figure BDA0004009983930000081
其中i是客户端id属于[1,10]。Step 1.1: The client receives the federated global model ω 5 delivered by the server. Each client of the federated system uses local data to update the local model
Figure BDA0004009983930000081
where i is the client id belonging to [1,10].

步骤2:客户端重要性评价Step 2: Client Importance Evaluation

步骤2.1:首先进行绝对重性检测,以客户端6为例,其在第5轮发生本地私有数据分布变化,使用本地模型

Figure BDA0004009983930000082
测试在第5轮本地私有增量数据集
Figure BDA0004009983930000083
上的准确率0.8,与在历史2轮本地私有增量数据集的并集
Figure BDA0004009983930000084
Figure BDA0004009983930000085
上的准确率0.85,进一步,使用两数据集准确率的差异的绝对值abs(0.80.85)反应当前轮次数据集与历史2轮数据集的差异为0.05,即客户端6本地数据在第5轮数据分布的绝对变化度量值为0.05。Step 2.1: First, perform absolute gravity detection. Taking client 6 as an example, its local private data distribution changes in the fifth round, using the local model
Figure BDA0004009983930000082
Tested on round 5 local private incremental dataset
Figure BDA0004009983930000083
Accuracy rate 0.8 on , combined with local private incremental datasets in historical 2 rounds
Figure BDA0004009983930000084
and
Figure BDA0004009983930000085
The accuracy rate above is 0.85. Further, the absolute value abs(0.80.85) of the difference between the accuracy rates of the two data sets reflects that the difference between the current round data set and the historical 2 round data sets is 0.05, that is, the local data of client 6 is at The absolute measure of change in the data distribution over 5 rounds was 0.05.

步骤2.2:计算客户端6的相对分布变化程度,客户端模型6在第5轮之前,于第3轮上传了本地模型,因此计算客户端第3轮到第5轮的累积梯度为

Figure BDA0004009983930000086
记做
Figure BDA0004009983930000087
同样的,计算客户端上一次上传时点的联邦全局模型ω5与ω3的累计梯度ω53记做
Figure BDA0004009983930000088
进一步的,计算
Figure BDA0004009983930000089
Figure BDA00040099839300000810
的余弦相似度为0.4,相对重要性度量为1-0.4=0.6。Step 2.2: Calculate the relative distribution change degree of client 6. The client model 6 uploaded the local model in the third round before the fifth round, so the cumulative gradient from the third round to the fifth round of the client is calculated as
Figure BDA0004009983930000086
remember to do
Figure BDA0004009983930000087
Similarly, calculate the cumulative gradient ω 53 of the federated global model ω 5 and ω 3 at the last upload time of the client.
Figure BDA0004009983930000088
Further, calculate
Figure BDA0004009983930000089
and
Figure BDA00040099839300000810
The cosine similarity of is 0.4, and the relative importance measure is 1-0.4=0.6.

步骤2.3:对客户端6在第5轮的重要性为其相对重要性与绝对重要性的乘积,即

Figure BDA00040099839300000811
Step 2.3: The importance of client 6 in round 5 is the product of relative importance and absolute importance, namely
Figure BDA00040099839300000811

步骤3:客户端通信概率控制Step 3: Client communication probability control

步骤3.1:基于客户端的重要性检测模块的度量结果,在每一轮联邦过程中,更新每个客户端j的通信概率为

Figure BDA00040099839300000812
其中超参数tau设置为2,Pfloor设置为5%。则客户端6在第5轮的通信概率为6%。Step 3.1: Based on the measurement results of the importance detection module of the client, in each round of the federation process, update the communication probability of each client j as
Figure BDA00040099839300000812
The hyperparameter tau is set to 2, and the P floor is set to 5%. Then the communication probability of client 6 in the fifth round is 6%.

步骤3.2:客户端6进行[0,1]的随机数,若随机值小于6%则上传本地模型的梯度信息到联邦服务器,否则不参与本轮上传。Step 3.2: Client 6 makes a random number of [0,1]. If the random value is less than 6%, it uploads the gradient information of the local model to the federation server, otherwise it does not participate in this round of uploading.

步骤4:联邦服务端聚合Step 4: Federated Server Aggregation

4.1联邦服务器将本轮上传的客户端梯度信息更新到联邦全局模型,假设本轮上传的客户端集合C∈[6,7,9],记客户端[6,7,9]的上传梯度分别为

Figure BDA0004009983930000091
则新一轮联邦全局模型
Figure BDA0004009983930000092
其中η为联邦全局模型的更新学习率,在实验中设置为0.01。4.1 The federation server updates the client gradient information uploaded in this round to the federated global model, assuming that the client set C ∈ [6,7,9] uploaded in this round, remember the uploaded gradients of the client [6,7,9] respectively for
Figure BDA0004009983930000091
Then a new round of federal global model
Figure BDA0004009983930000092
where η is the update learning rate of the federated global model, which is set to 0.01 in the experiment.

4.2联邦服务器将联邦全局模型ω6下发给所有客户端,进入下一轮联邦过程。4.2 The federation server sends the federated global model ω 6 to all clients and enters the next round of federation process.

通过实验证明,在20%客户端分布随时间变化的场景中,本发明系统对比Fed-average与FedPNS等基线方法,能以平均30%客户端上传的通信成本下达到与Fed-average和FedPNS方法平均40%客户端参与联邦上传相同的性能,即在性能相同的情况下降低10%的上行通信开销。有效提升了联邦通信效率,实现了在时空异构场景上的轻量化联邦学习。It has been proved by experiments that in the scenario where 20% of the client distribution changes with time, the system of the present invention compares the baseline methods such as Fed-average and FedPNS, and can reach the Fed-average and FedPNS method with an average communication cost of 30% of client uploads. On average, 40% of the clients participate in the federation to upload the same performance, that is, the uplink communication overhead is reduced by 10% under the same performance. It effectively improves the efficiency of federated communication and realizes lightweight federated learning in spatiotemporal heterogeneous scenarios.

综上,本发明提出了一种面向时空数据异构场景的轻量化联邦学习系统及方法,在智能医疗等物联网场景中,利用客户端的私有数据分布变化信息度量客户端的重要性,在联邦过程中动态的按重要性调整客户端的通信概率,降低联邦系统的通信开销各图像采集分析终端利用其私有病例数据分布的变化信息以及图像采集分析终端私有检测模型的梯度信息,在每一轮联邦过程中计算其重要性。并根据图像采集分析终端重要度量结果,动态的调整每一轮联邦过程图像采集分析终端参与联邦通信的概率。通过将通信需求分配给更重要的图像采集分析终端,降低低重要性图像采集分析终端的通信量,实现以更少的联邦通信成本达到相同联邦学习性能,有效的降低联邦学习通信量。To sum up, the present invention proposes a lightweight federated learning system and method for spatio-temporal data heterogeneous scenarios. In IoT scenarios such as smart medical care, the client’s private data distribution change information is used to measure the importance of the client. In the federated process Dynamically adjust the communication probability of the client according to the importance, and reduce the communication overhead of the federation system. Each image collection and analysis terminal uses the change information of its private case data distribution and the gradient information of the private detection model of the image collection and analysis terminal, in each round of the federation process calculate its importance. And according to the important measurement results of the image collection and analysis terminal, dynamically adjust the probability of each round of federation process image collection and analysis terminal participating in the federation communication. By allocating communication needs to more important image acquisition and analysis terminals, reducing the communication traffic of low-importance image acquisition and analysis terminals, achieving the same federated learning performance with less federated communication costs, and effectively reducing federated learning traffic.

以上实施例仅用以说明本发明的技术方案,而非对其限制;尽管参照前述实施例对本发明进行了详细的说明,本领域的普通技术人员应当理解:其依然可以对前述各实施例所记载的技术方案进行修改,或者对其中部分技术特征进行等同替换,但这些修改或者替换,并不使相应技术方案的本质脱离本发明各实施例技术方案的精神和范围。The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still be described in the foregoing embodiments The recorded technical solutions are modified, or some of the technical features are equivalently replaced, but these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims (6)

1. A lightweight federated learning method for space-time data heterogeneous scenes is characterized by comprising the following steps:
s1, local updating of a client: after receiving the latest federated global model, updating the federated global model into a localization model which is more consistent with local private data distribution by using local private data of the client;
s2, client importance evaluation: measuring the importance of the client participating in the federal communication on the global model of the federation by using the distribution change degree of private data and the consistency of the gradient information of the client model and the gradient information of the global model;
s3, controlling the communication probability of the client: dynamically adjusting the communication probability of the client according to the importance of the client obtained in the step S2, judging whether to upload according to the probability, if so, entering the step S4, and if not, issuing the federal global model to all the clients;
s4, federal service side aggregation: updating the global federal model by using the gradient information uploaded by the clients participating in the federal update in the current round, issuing the updated global federal model to all the clients, and entering the next federal process until the preset federal turn is reached.
2. The lightweight federal learning method for spatio-temporal data heterogeneous scenes as claimed in claim 1, wherein the specific process of the step S1 is as follows: local to the client, in the federal global model ω t On the basis, by a gradient descent iterative algorithm, using a private data set of the client i in t turns
Figure FDA0004009983920000011
Continue training to obtain the localModel (model)
Figure FDA0004009983920000012
The optimization objective function of the client is
Figure FDA0004009983920000013
Wherein n is
Figure FDA0004009983920000014
And f is a loss function of the client i.
3. The lightweight federal learning method for spatio-temporal data heterogeneous scenes as claimed in claim 1, wherein the specific process of step S2 is as follows:
s21, the model obtained after the local update of the t round of the client i is
Figure FDA0004009983920000015
First, a model is calculated
Figure FDA0004009983920000016
Top-1 accuracy on recent data fragmentation
Figure FDA0004009983920000017
Second calculation model
Figure FDA0004009983920000018
Accuracy on the last n rounds of data slicing
Figure FDA0004009983920000019
Finally, calculating the absolute value of the difference of the accuracy rates of the data at two different time ends on the same model as
Figure FDA00040099839200000110
Reflect the absolute distribution change of the local data of the client, and record it as
Figure FDA00040099839200000111
S22, caching the local model uploaded to the federal server last time by the client
Figure FDA00040099839200000112
Compare it with local model of the current round
Figure FDA00040099839200000113
Obtaining gradient information of the client end which is not uploaded
Figure FDA00040099839200000114
Comparing the received federal global model omega which is calculated by the client after last uploading s With the latest federal global model omega t Obtaining the perceptual gradient information of the global model of the federation
Figure FDA0004009983920000021
By calculating the cosine similarity of the gradient information of the client and the gradient information of the global federated model
Figure FDA0004009983920000022
The relative difference between the gradient to be uploaded at the client and the perceived gradient of the global federal model is measured and recorded as
Figure FDA0004009983920000023
S23, combining the absolute change AC and the relative change RC in a multiplication mode, and taking the combination as the importance measurement of the client i in the round t:
Figure FDA0004009983920000024
4. the lightweight federal learning method for spatio-temporal data heterogeneous scenes as claimed in claim 1, wherein the specific process of step S3 is:
s31, customerThe importance metric of the client is input into a communication probability mapping module, and the communication probability of the client i is updated to
Figure FDA0004009983920000025
The tau control federal system distributes communication resources for the time change of data distribution, and reflects the sensitivity degree of the distribution change; p is floor The basic communication probability of the client is controlled, the uplink probability of the high-importance client is dynamically increased, and the uplink probability of the low-importance client is reduced;
and S32, the communication probability control module generates a client-side unique heat vector to be uploaded according to the probability in each round of federal process, wherein one dimension value is 1, and the other dimension values are 0, and the corresponding client side is required to upload the gradient information of the local model.
5. The lightweight federal learning method for spatio-temporal data heterogeneous scenes as claimed in claim 1, wherein the specific process of step S4 is:
s41, uploading the gradient accumulated gradient from the S moment to the t moment in the t round by the selected client i
Figure FDA0004009983920000026
S42, the client set uploaded in the current round is C t The historical federal global model is ω t Updating the accumulated gradient information uploaded to the client to a federal global model by a learning rate eta
Figure FDA0004009983920000027
And S43, issuing the updated global federated model to all clients.
6. A lightweight federated learning system oriented to spatio-temporal data heterogeneous scenarios, comprising the following modules to implement the method of any one of claims 1-5:
the client local updating module is used for updating the federal global model into a localization model which is more in line with the distribution of local private data by using local private data of the client after receiving the latest federal global model;
the client importance evaluation module is used for measuring the importance of the client participating in the federal communication on the federal global model by utilizing the distribution change degree of private data and the consistency of the gradient information of the client model and the gradient information of the global model;
the client communication probability control module dynamically adjusts the client communication probability according to the client importance obtained by the client importance evaluation module, judges whether to upload according to the probability, enters the federal service end aggregation module if the client communication probability is uploaded, and issues the federal global model to all clients if the client communication probability control module does not upload the client communication probability;
and the federal server side aggregation module is used for updating the federal global model by using the gradient information uploaded by the clients participating in the federal update in the current round, issuing the updated federal global model to all the clients and entering the next federal process until the preset federal turn is reached.
CN202211649919.4A 2022-12-21 2022-12-21 Lightweight federated learning method and system for space-time data heterogeneous scene Pending CN115775025A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211649919.4A CN115775025A (en) 2022-12-21 2022-12-21 Lightweight federated learning method and system for space-time data heterogeneous scene

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211649919.4A CN115775025A (en) 2022-12-21 2022-12-21 Lightweight federated learning method and system for space-time data heterogeneous scene

Publications (1)

Publication Number Publication Date
CN115775025A true CN115775025A (en) 2023-03-10

Family

ID=85393253

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211649919.4A Pending CN115775025A (en) 2022-12-21 2022-12-21 Lightweight federated learning method and system for space-time data heterogeneous scene

Country Status (1)

Country Link
CN (1) CN115775025A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115018085A (en) * 2022-05-23 2022-09-06 郑州大学 A Federated Learning Participating Device Selection Method for Data Heterogeneity

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115018085A (en) * 2022-05-23 2022-09-06 郑州大学 A Federated Learning Participating Device Selection Method for Data Heterogeneity

Similar Documents

Publication Publication Date Title
CN115587633A (en) A Personalized Federated Learning Method Based on Parameter Hierarchy
Cao et al. C2S: Class-aware client selection for effective aggregation in federated learning
CN116306910A (en) Fair privacy calculation method based on federal node contribution
Zhang et al. An edge based federated learning framework for person re-identification in UAV delivery service
CN116384504A (en) Federal migration learning system
CN116702881A (en) Multilayer federal learning scheme based on sampling aggregation optimization
CN112507893A (en) Distributed unsupervised pedestrian re-identification method based on edge calculation
CN116386853A (en) A deep separable convolution dual aggregation federated learning method for smart healthcare
CN115775025A (en) Lightweight federated learning method and system for space-time data heterogeneous scene
CN117496335A (en) Intelligent identification method and system for skin lesions based on federal semi-supervised learning
CN115563519A (en) Federal contrast clustering learning method and system for non-independent same-distribution data
CN113918829A (en) Content caching and recommending method based on federal learning in fog computing network
Yang et al. Efficient knowledge management for heterogeneous federated continual learning on resource-constrained edge devices
Yang et al. Hypernetworks-based hierarchical federated learning on hybrid non-IID datasets for digital twin in industrial IoT
CN115129888A (en) Active content caching method based on network edge knowledge graph
CN107844532A (en) Based on MapReduce and the extensive nearest Neighbor for arranging Thiessen polygon
CN115359298A (en) A Federated Meta-Learning Image Classification Method Based on Sparse Neural Networks
CN113361598B (en) Model training method based on distributed learning, server and distributed system
CN113822291B (en) Image processing method, device, equipment and storage medium
Li et al. Client-edge-cloud hierarchical federated learning based on generative adversarial networks
CN113037872A (en) Caching and prefetching method based on Internet of things multi-level edge nodes
CN118862948A (en) Personalized federated learning method based on knowledge distillation and sharpness-aware minimization
CN118839750A (en) Clustering federation learning method based on data characterization optimization and related equipment
CN107766496A (en) Based on MapReduce and the extensive NN Query system for arranging Thiessen polygon
CN115481735A (en) Aggregation Optimization System of Attention Mechanism Based on Federated Learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination