CN115278709A - Communication optimization method based on federal learning - Google Patents

Communication optimization method based on federal learning Download PDF

Info

Publication number
CN115278709A
CN115278709A CN202210906790.4A CN202210906790A CN115278709A CN 115278709 A CN115278709 A CN 115278709A CN 202210906790 A CN202210906790 A CN 202210906790A CN 115278709 A CN115278709 A CN 115278709A
Authority
CN
China
Prior art keywords
gradient
local
model
matrix
client
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210906790.4A
Other languages
Chinese (zh)
Other versions
CN115278709B (en
Inventor
张盼
许春根
徐磊
梅琳
窦本年
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University of Science and Technology
Original Assignee
Nanjing University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing University of Science and Technology filed Critical Nanjing University of Science and Technology
Priority to CN202210906790.4A priority Critical patent/CN115278709B/en
Publication of CN115278709A publication Critical patent/CN115278709A/en
Application granted granted Critical
Publication of CN115278709B publication Critical patent/CN115278709B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W16/00—Network planning, e.g. coverage or traffic planning tools; Network deployment, e.g. resource partitioning or cells structures
    • H04W16/22—Traffic simulation tools or models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W24/00—Supervisory, monitoring or testing arrangements
    • H04W24/02—Arrangements for optimising operational condition

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Evolutionary Computation (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Abstract

The invention provides a communication optimization method based on federal learning, belonging to the technical field of federal machine learning, comprising the following steps: acquiring local gradient core characteristics uploaded by each client; the local gradient core characteristics refer to data extracted by preprocessing target data by each client, training a local model and performing singular value decomposition on the local model; performing sketch gradient compression and sketch gradient aggregation on the local gradient core characteristics, extracting the first k sparse parameter matrixes, executing a federal average aggregation algorithm on the first k sparse parameter matrixes to obtain a global model, and sending the global model to each client; and calculating the accumulated gradient error value and the gradient error compensation value of each client, adaptively adjusting the weight of the gradient of the local gradient core characteristic to be compensated and the weight of the local gradient core characteristic in the process of sketch gradient aggregation, and then iterating. The method can effectively compress the gradient of the model, reduce communication overhead generated in model training and improve the accuracy of model prediction.

Description

Communication optimization method based on federal learning
Technical Field
The invention relates to the technical field of federal machine learning, in particular to a communication optimization method based on federal learning.
Background
The existing federal learning generally uploads the intermediate parameters of model training to a parameter server to realize model aggregation, and two problems exist in the process: firstly, large-scale distributed training needs a large amount of communication bandwidth for gradient exchange, which limits the expandability of multi-node training; the second is the need for expensive high bandwidth network infrastructure.
When distributed training is performed on a mobile device, there is higher network latency, lower system throughput, and intermittent poor connectivity.
In order to solve the above problem, if the traditional SVD singular value decomposition method is directly adopted, although the accuracy of model prediction is improved, the gradient compression effect is not ideal. If the sketch compression method is adopted, although the gradient compression effect is ideal, the accuracy of model prediction is reduced.
Disclosure of Invention
The invention aims to provide a communication optimization method based on federal learning, which can effectively compress model gradient, reduce communication overhead generated in model training and improve the accuracy of model prediction.
In order to achieve the purpose, the invention adopts the technical scheme that:
a communication optimization method based on federal learning comprises the following steps: s1, acquiring local gradient core characteristics uploaded by each client; the local gradient core characteristics refer to model gradients extracted by preprocessing target data by each client to train a local model and performing singular value decomposition on the local model; the local model is a federal model; s2, carrying out sketch gradient compression on the local gradient core characteristics, carrying out sketch gradient aggregation, extracting the first k sparse parameter matrixes, executing a Federal average aggregation algorithm on the first k sparse parameter matrixes to obtain a global model, and distributing the global model to each client; s3, calculating the accumulated gradient error value and the gradient error compensation value of each client, adaptively adjusting the gradient of the local gradient core characteristic to be compensated and the weight of the local gradient core characteristic in the process of sketch gradient aggregation, and then iteratively executing S1 to S3.
Further, preprocessing the target data includes: dividing target data into a plurality of data sets, and executing uniform random sampling operation to select the batch size of the data sets in each iteration to obtain a target data set; performing paradigm boundary calculation on the target data set; acquiring initialized global model parameters, training a local model through a target data set after the normal form boundary calculation, and calculating the local gradient of the local model; and cutting the local gradient to obtain a gradient matrix of the local model.
Further, the local model is subjected to singular value decomposition as: and each client executes a rapid gradient descent algorithm, decomposes the gradient matrix of the local model into a left singular vector, a right singular vector and a diagonal matrix containing the gradient, and obtains the core characteristic of the local gradient.
Further, each client executes a fast gradient descent algorithm, decomposes the gradient matrix of the local model into a left singular vector, a right singular vector and a diagonal matrix containing gradients, and obtains local gradient core features, including: storing data of a gradient matrix of a local model in a zero matrix in sequence, and compressing until zero-value rows do not exist in the zero matrix to obtain a first matrix; constructing a sub-sampled random Hadamard transform matrix, calculating the product of the sub-sampled random Hadamard transform matrix and a first matrix, and compressing the first matrix from m to
Figure BDA0003772780860000021
Obtaining a second matrix; and executing an SVD algorithm on the second matrix, decomposing the gradient matrix of the local model into a left singular vector, a right singular vector and a diagonal matrix containing gradients, and obtaining the core characteristics of the local gradients.
Further, S3 includes: calculating a local model loss value of each client in the current iteration process; calculating the cross entropy of the loss value of the local model, and calculating the quality evaluation weight of the local model according to the cross entropy; calculating local gradient errors generated by draft gradient compression in the draft gradient aggregation process to obtain accumulated gradient error values and gradient error compensation values of all clients; adjusting the weight of the local gradient core characteristics of each client for sketch gradient aggregation according to the quality evaluation weight and the accumulated gradient error value of the local model of each client; and adaptively adjusting the gradient to be compensated of the local gradient core characteristic of the whole client according to the gradient error compensation value, and entering the next iteration.
Drawings
The invention and its features, aspects and advantages will become more apparent from reading the following detailed description of non-limiting embodiments with reference to the accompanying drawings. Like reference symbols in the various drawings indicate like elements. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the invention.
FIG. 1 is a general flow chart of a federated learning-based communication optimization method provided by the present invention;
FIG. 2 is a detailed flowchart of a preprocessing algorithm in a federated learning-based communication optimization method provided by the present invention;
FIG. 3 is a detailed flowchart of a fast gradient descent algorithm in a communication optimization method based on federated learning according to the present invention;
fig. 4 is a specific flowchart for performing gradient error compensation in the communication optimization method based on federal learning according to the present invention.
Detailed Description
The invention will be further described with reference to the following drawings and specific examples, which should not be construed as limiting the invention thereto.
The method aims to solve the problems that in the prior art, when the intermediate parameters of model training are uploaded to a parameter server to realize model aggregation in a Federal learning algorithm, a traditional SVD singular value decomposition method is directly adopted, although the accuracy of model prediction is improved, the gradient compression effect is not ideal; the method is characterized in that a sketch compression method is directly adopted, although the gradient compression effect is ideal, the problem of the accuracy of model prediction is reduced, therefore, as shown in fig. 1, the invention provides a communication optimization method based on federal learning, in particular to a mobile edge network communication optimization method, aiming at the problems that when distributed training is carried out under a mobile edge network, the resources of a mobile terminal are limited, and the training can bring higher network delay, lower system throughput and intermittent network connection failure, the communication optimization is carried out on the communication optimization, and the huge communication overhead generated in the model training process is reduced.
Firstly, each client needs to preprocess local target data, and then trains out a local model, wherein the local model is a federal model; as shown in fig. 2, the steps specifically include:
splitting target data into minipatch datasets
Figure BDA0003772780860000031
And executing uniform random sampling operation to select the batch size of the data set in each iteration to obtain a target data set d'i＝di⊙Sc(ii) a Wherein i is the ith client, diIs the data of the ith client, c is the number of matrix columns, ScInputting a random uniform sampling matrix with the same dimension as the sensitive original data;
since each client is to be in target dataset d'iRun a Federal learning model to extract features, d'il＝F(d′i) To limit a certain global sensitivity, the target data set d'iPerforming normal form boundary calculations, i.e.
Figure BDA0003772780860000041
Wherein,
Figure BDA0003772780860000045
is the minimum data batch size; dlAn infinite norm representing a first level output of the neural network model; d 'can be known from the formula'lHas an upper limit of Sf＝max||f(d)-f(d′)||2(ii) a Wherein f is a real-valued function, provided that
Figure BDA0003772780860000046
dlThe sensitivity of the data can be protected, and the sensitive data can be protected from being leaked to a certain extent;
the global model parameters w are then initializedglobalAnd distributing the data to each client, training a local model by each client according to the target data set calculated by the paradigm boundary, and calculating the local gradient g of the local model(i)(ii) a Clipping local gradients, i.e.
Figure BDA0003772780860000042
So as to prevent gradient explosion and finally obtain the gradient matrix A of the local model.
Then each client executes a rapid gradient descent algorithm, as shown in fig. 3, a gradient matrix of the local model is decomposed into a left singular vector, a right singular vector and a diagonal matrix containing gradients to approximately describe matrix information of a certain percentage of the gradient matrix, and then the gradients are compressed in a matrix form to obtain local gradient core characteristics, so that the high communication cost generated by federal training is reduced; the method specifically comprises the following steps:
collecting gradient matrix A epsilon R of local model in sequencem×nN data points in the mth row and stored in the zero matrix F e Rm×nCompressing all zero-value rows in the zero matrix to obtain a first matrix F';
when judging that zero-value rows do not exist in the F, constructing a sub-sampling random Hadamard transform matrix phi = N.H.D; wherein N ∈ Rq×mRepresents a scaled random matrix that is uniformly sampled per row without replacement from m rows of an m identity matrix, rescaled to
Figure BDA0003772780860000043
D represents an m x m diagonal matrix with elements of i.i.d, being a random variable (equal probability-1 or 1); h belongs to { +/-1 }m×mExpressed as Hadamard matrix (Hadamard) meaning hij＝(-1)<i-1，j-1>，<i-1，j-1>Is the dot product of the b-bit binary vectors of integers i-1 and j-1,
Figure BDA0003772780860000047
the i, j represents the ith row and the jth column) Hadamard matrix can also be expressed as
Figure BDA0003772780860000044
Where m is the size of the matrix;
computing the product of the sub-sampled random Hadamard transform matrix Φ and the first matrix F 'to compress the size of F' from m to
Figure BDA0003772780860000051
And connecting the new compressed gradient and the shrunk gradient to obtain a second matrix B epsilon Rk×n；
Performing SVD algorithm on the second matrix according to [ U, lambda, V]= SVD (B) decompose gradient matrix of local model into left singular vector, right singular vector and diagonal matrix containing gradient and pass
Figure BDA0003772780860000052
Compressing the original information to the front of B
Figure BDA0003772780860000053
And improving the communication efficiency of uploading the local model parameters under the condition that the influence on the local gradient kernel is small.
Performing sketch gradient compression on local gradient core characteristics of each client, performing sketch gradient aggregation, extracting the first k sparse parameter matrixes, performing a federal average aggregation algorithm on the first k sparse parameter matrixes to obtain a global model, and distributing the global model to each client;
then, as shown in fig. 4, calculating an accumulated gradient error value and a gradient error compensation value of each client, adaptively adjusting the gradient of the local gradient core feature to be compensated and the weight of the local gradient core feature during sketch gradient aggregation, and substituting the weights into the next iterative computation global model; the method specifically comprises the following steps:
calculating the locality of each client in the current iteration processLoss value of model
Figure BDA0003772780860000054
Wherein, DOiIs the ith client(s),
Figure BDA0003772780860000055
is a parameter of the local model of the ith client in the t-th iteration;
calculating the cross entropy H (f (x) of the local model loss valuei)，yi)＝-∑xiyilogf(xi) Evaluating local model penalty values by cross entropy
Figure BDA0003772780860000056
If the cross entropy H (f (x)i)，yi) The smaller the value is, the closer the local model prediction result is to the real result is, which indicates that the model training quality is higher;
then, the quality evaluation weight of the local model is calculated according to the cross entropy
Figure BDA0003772780860000057
Wherein (x)i，yi) Is the local data tag, x, of the ith clientiIs input data, yiIs the desired output, f (x)i) Is the local model prediction result of the ith client;
aggregating the quality assessment weights of the clients, client D, each time in a sketch gradient manneriThe local gradient error generated by the compression of the gradient of the sketch in the gradient aggregation process of the sketch can be used as the accumulated gradient error value
Figure BDA0003772780860000058
It is shown that the process of the present invention,
Figure BDA0003772780860000061
the accumulated gradient error value of the ith client in the t-th iteration is represented, beta is an error factor, and the accumulated gradient error value can be continuously updated along with the iteration; then according to
Figure BDA0003772780860000062
Wherein α is a compensation coefficient;
Figure BDA0003772780860000063
is the gradient after SVD and Sketch;
adjusting the weight of the local gradient core characteristics of each client for sketch gradient aggregation according to the quality evaluation weight and the accumulated gradient error value of the local model of each client; according to the gradient error compensation value to adaptively adapt to the gradient to be compensated of the local gradient core characteristic of the whole client, optimal parameters alpha and beta are selected and substituted into the next iterative computation, so that the accuracy of the global model is improved, and the optimal parameters alpha and beta are expressed as the gradient to be compensated in the global model updating computation process
Figure BDA0003772780860000064
Where n is the number of clients participating in model training, ψ is the number of high quality clients selected uniformly and randomly, η is the local model learning rate, Pi(Di) Is a client DiThe quality evaluation weight of the local model in the ith model aggregation, wherein the model aggregation is to perform federal averaging on uploaded gradient updates of all clients and is used for updating the global model);
Figure BDA0003772780860000065
is the local gradient that client i did not compensate in the t-th iteration, alpha is the compensation coefficient,
Figure BDA0003772780860000066
is the cumulative gradient error value of the ith client in the tth iteration.
In summary, according to the communication optimization method based on federal learning provided by the present invention, each client independently trains a local model, then performs Singular Value Decomposition (SVD) on the local model, extracts local gradient core features, sends the local gradient core features to a parameter server, performs Sketch (Sketch) gradient compression by the parameter server, then performs Sketch (Sketch) gradient aggregation, extracts the first k approximate gradient values from the aggregated parameters, performs the federal average aggregation algorithm on the first k approximate gradient values to obtain a global model, and sends the global model to each client to perform the next iteration process. And aiming at local gradient errors generated after the Sketch gradient lossy compression, self-adaptive gradient error accumulation and gradient error compensation are supported, the gradient to be compensated is self-adaptively adjusted, the gradient is added into the next iteration, the model gradient can be effectively compressed through SVD and Sketch compression, the communication overhead generated in model training is reduced, the gradient errors between SVD and Sketch operation and SVD operation are only executed due to the combination of SVD and Sketch operation are solved through the gradient error compensation, and therefore the accuracy of global model prediction is improved. The invention can greatly reduce the communication overhead between the client and the server in the whole training process, accelerate the convergence of the model and ensure that the accuracy of the model prediction is almost the same as the original prediction accuracy.
The above description is of the preferred embodiment of the invention; it is to be understood that the invention is not limited to the particular embodiments described above, in that devices and structures not described in detail are understood to be implemented in a manner common in the art; any person skilled in the art can make many possible variations and modifications, or modify equivalent embodiments, without departing from the technical solution of the invention, without affecting the essence of the invention; therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention are within the scope of the technical solution of the present invention, unless the technical essence of the present invention is not departed from the content of the technical solution of the present invention.

Claims (5)

1. A communication optimization method based on federal learning is characterized by comprising the following steps:
s1, acquiring local gradient core characteristics uploaded by each client; the local gradient core characteristics refer to model gradients extracted by preprocessing target data by each client, training a local model and performing singular value decomposition on the local model; the local model is a federal model;
s2, performing sketch gradient compression on the local gradient core features, performing sketch gradient aggregation, extracting the first k sparse parameter matrixes, performing a federal average aggregation algorithm on the first k sparse parameter matrixes to obtain a global model, and distributing the global model to each client;
s3, calculating accumulated gradient error values and gradient error compensation values of all the clients, adaptively adjusting the gradient of the local gradient core characteristics to be compensated and the weight of the local gradient core characteristics during sketch gradient aggregation, and then iteratively executing S1 to S3.
2. The method of claim 1, wherein the preprocessing the target data comprises:
dividing target data into a plurality of data sets, and executing uniform random sampling operation to select the batch size of the data sets in each iteration to obtain a target data set;
performing paradigm boundary calculation on the target data set;
acquiring initialized global model parameters, training a local model through a target data set after the normal form boundary calculation, and calculating the local gradient of the local model;
and cutting the local gradient to obtain a gradient matrix of the local model.
3. The federal learning-based communication optimization method as claimed in claim 2, wherein the singular value decomposition of the local model is as follows:
and each client executes a rapid gradient descent algorithm, and decomposes the gradient matrix of the local model into a left singular vector, a right singular vector and a diagonal matrix containing gradients to obtain local gradient core characteristics.
4. The federal learning-based communication optimization method of claim 3, wherein each client executes a fast gradient descent algorithm to decompose a gradient matrix of the local model into a left singular vector, a right singular vector and a diagonal matrix containing gradients, so as to obtain local gradient core features, and the method includes:
storing data of a gradient matrix of a local model in a zero matrix in sequence, and compressing until zero-value rows do not exist in the zero matrix to obtain a first matrix;
constructing a sub-sampled random Hadamard transform matrix, calculating the product of the sub-sampled random Hadamard transform matrix and a first matrix, compressing the first matrix from m to
Figure FDA0003772780850000021
Obtaining a second matrix;
and executing an SVD algorithm on the second matrix, decomposing the gradient matrix of the local model into a left singular vector, a right singular vector and a diagonal matrix containing gradients, and obtaining the core characteristics of the local gradients.
5. The method according to claim 4, wherein the S3 comprises:
calculating a local model loss value of each client in the current iteration process;
calculating the cross entropy of the loss value of the local model, and calculating the quality evaluation weight of the local model according to the cross entropy;
calculating local gradient errors generated by the gradient compression of the sketch in the gradient aggregation process of the sketch to obtain accumulated gradient error values and gradient error compensation values of all clients;
adjusting the weight of the local gradient core characteristics of each client for sketch gradient aggregation according to the quality evaluation weight and the accumulated gradient error value of the local model of each client; and adaptively adjusting the gradient to be compensated of the local gradient core characteristic of the whole client according to the gradient error compensation value, and entering the next iteration.
CN202210906790.4A 2022-07-29 2022-07-29 Communication optimization method based on federal learning Active CN115278709B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210906790.4A CN115278709B (en) 2022-07-29 2022-07-29 Communication optimization method based on federal learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210906790.4A CN115278709B (en) 2022-07-29 2022-07-29 Communication optimization method based on federal learning

Publications (2)

Publication Number Publication Date
CN115278709A true CN115278709A (en) 2022-11-01
CN115278709B CN115278709B (en) 2024-04-26

Family

ID=83772330

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210906790.4A Active CN115278709B (en) 2022-07-29 2022-07-29 Communication optimization method based on federal learning

Country Status (1)

Country Link
CN (1) CN115278709B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117216596A (en) * 2023-08-16 2023-12-12 中国人民解放军总医院 Federal learning optimization communication method, system and storage medium based on gradient clustering
CN117692939A (en) * 2024-02-02 2024-03-12 南京邮电大学 Client scheduling method in dynamic communication environment

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190087713A1 (en) * 2017-09-21 2019-03-21 Qualcomm Incorporated Compression of sparse deep convolutional network weights
US20200117580A1 (en) * 2018-04-22 2020-04-16 Sas Institute Inc. Validation Sets for Machine Learning Algorithms
US20200142948A1 (en) * 2018-11-07 2020-05-07 Samsung Electronics Co., Ltd. System and method for cached convolution calculation
CN111507481A (en) * 2020-04-17 2020-08-07 腾讯科技（深圳）有限公司 Federated learning system
CN112199702A (en) * 2020-10-16 2021-01-08 鹏城实验室 Privacy protection method, storage medium and system based on federal learning
CN112288097A (en) * 2020-10-29 2021-01-29 平安科技（深圳）有限公司 Federal learning data processing method and device, computer equipment and storage medium
CN112862011A (en) * 2021-03-31 2021-05-28 中国工商银行股份有限公司 Model training method and device based on federal learning and federal learning system
CN113222179A (en) * 2021-03-18 2021-08-06 北京邮电大学 Federal learning model compression method based on model sparsification and weight quantization
CN113645197A (en) * 2021-07-20 2021-11-12 华中科技大学 Decentralized federal learning method, device and system
CN114339252A (en) * 2021-12-31 2022-04-12 深圳大学 Data compression method and device

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190087713A1 (en) * 2017-09-21 2019-03-21 Qualcomm Incorporated Compression of sparse deep convolutional network weights
US20200117580A1 (en) * 2018-04-22 2020-04-16 Sas Institute Inc. Validation Sets for Machine Learning Algorithms
US20200142948A1 (en) * 2018-11-07 2020-05-07 Samsung Electronics Co., Ltd. System and method for cached convolution calculation
CN111507481A (en) * 2020-04-17 2020-08-07 腾讯科技（深圳）有限公司 Federated learning system
CN112199702A (en) * 2020-10-16 2021-01-08 鹏城实验室 Privacy protection method, storage medium and system based on federal learning
CN112288097A (en) * 2020-10-29 2021-01-29 平安科技（深圳）有限公司 Federal learning data processing method and device, computer equipment and storage medium
CN113222179A (en) * 2021-03-18 2021-08-06 北京邮电大学 Federal learning model compression method based on model sparsification and weight quantization
CN112862011A (en) * 2021-03-31 2021-05-28 中国工商银行股份有限公司 Model training method and device based on federal learning and federal learning system
CN113645197A (en) * 2021-07-20 2021-11-12 华中科技大学 Decentralized federal learning method, device and system
CN114339252A (en) * 2021-12-31 2022-04-12 深圳大学 Data compression method and device

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
LAIZHONG CUI;XIAOXIN SU;YIPENG ZHOU;LEI ZHANG: ""ClusterGrad: Adaptive Gradient Compression by Clustering in Federated Learning"", 《GLOBECOM 2020 - 2020 IEEE GLOBAL COMMUNICATIONS CONFERENCE》, 25 January 2021 (2021-01-25) *
P. RAVICHANDRAN;C. SARAVANAKUMAR;J. DAFNI ROSE;M. VIJAYAKUMAR;V. MUTHU LAKSHMI: ""Efficient Multilevel Federated Compressed Reinforcement Learning of Smart Homes Using Deep Learning Methods"", 《2021 INTERNATIONAL CONFERENCE ON INNOVATIVE COMPUTING, INTELLIGENT COMMUNICATION AND SMART ELECTRICAL SYSTEMS (ICSES)》, 16 December 2021 (2021-12-16) *
杨阳: "物联网中数据压缩与资源分配的研究", 《中国博士学位论文全文数据库（电子期刊）》, 15 May 2020 (2020-05-15) *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117216596A (en) * 2023-08-16 2023-12-12 中国人民解放军总医院 Federal learning optimization communication method, system and storage medium based on gradient clustering
CN117216596B (en) * 2023-08-16 2024-04-30 中国人民解放军总医院 Federal learning optimization communication method, system and storage medium based on gradient clustering
CN117692939A (en) * 2024-02-02 2024-03-12 南京邮电大学 Client scheduling method in dynamic communication environment
CN117692939B (en) * 2024-02-02 2024-04-12 南京邮电大学 Client scheduling method in dynamic communication environment

Also Published As

Publication number Publication date
CN115278709B (en) 2024-04-26

Similar Documents

Publication Publication Date Title
CN115278709A (en) Communication optimization method based on federal learning
CN111555781B (en) Large-scale MIMO channel state information compression and reconstruction method based on deep learning attention mechanism
CN111898703B (en) Multi-label video classification method, model training method, device and medium
CN111259917B (en) Image feature extraction method based on local neighbor component analysis
CN114332500B (en) Image processing model training method, device, computer equipment and storage medium
CN115829027A (en) Comparative learning-based federated learning sparse training method and system
CN115587633A (en) Personalized federal learning method based on parameter layering
CN113658130A (en) No-reference screen content image quality evaluation method based on dual twin network
CN116168197A (en) Image segmentation method based on Transformer segmentation network and regularization training
CN115865145A (en) Large-scale MIMO channel state information feedback method based on Transformer
CN114626550A (en) Distributed model collaborative training method and system
CN117236201B (en) Diffusion and ViT-based downscaling method
CN116797850A (en) Class increment image classification method based on knowledge distillation and consistency regularization
CN114677545B (en) Lightweight image classification method based on similarity pruning and efficient module
CN115952860A (en) Heterogeneous statistics-oriented clustering federal learning method
CN115909441A (en) Face recognition model establishing method, face recognition method and electronic equipment
CN114677535A (en) Training method of domain-adaptive image classification network, image classification method and device
CN112116062A (en) Multilayer perceptron nonlinear compression method based on tensor string decomposition
Limmer et al. Optimal deep neural networks for sparse recovery via Laplace techniques
CN117222005B (en) Fingerprint positioning method, fingerprint positioning device, electronic equipment and storage medium
CN110111326B (en) Reconstructed image quality evaluation method based on ERT system
CN115019079B (en) Method for accelerating deep learning training by distributed outline optimization for image recognition
CN114581946B (en) Crowd counting method and device, storage medium and electronic equipment
CN112598130B (en) Soil moisture data reconstruction method based on self-encoder and singular value threshold and computer readable storage medium
Li et al. HyperFeel: An Efficient Federated Learning Framework Using Hyperdimensional Computing

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant