CN115278709A - Communication optimization method based on federal learning - Google Patents
Communication optimization method based on federal learning Download PDFInfo
- Publication number
- CN115278709A CN115278709A CN202210906790.4A CN202210906790A CN115278709A CN 115278709 A CN115278709 A CN 115278709A CN 202210906790 A CN202210906790 A CN 202210906790A CN 115278709 A CN115278709 A CN 115278709A
- Authority
- CN
- China
- Prior art keywords
- gradient
- local
- model
- matrix
- client
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000004891 communication Methods 0.000 title claims abstract description 26
- 238000005457 optimization Methods 0.000 title claims abstract description 18
- 230000002776 aggregation Effects 0.000 claims abstract description 24
- 238000004220 aggregation Methods 0.000 claims abstract description 24
- 238000012549 training Methods 0.000 claims abstract description 20
- 230000006835 compression Effects 0.000 claims abstract description 16
- 238000007906 compression Methods 0.000 claims abstract description 16
- 230000008569 process Effects 0.000 claims abstract description 12
- 238000000354 decomposition reaction Methods 0.000 claims abstract description 8
- 238000007781 pre-processing Methods 0.000 claims abstract description 6
- 239000011159 matrix material Substances 0.000 claims description 59
- 239000013598 vector Substances 0.000 claims description 17
- 238000013441 quality evaluation Methods 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000005070 sampling Methods 0.000 claims description 5
- 238000010801 machine learning Methods 0.000 abstract description 2
- 230000000694 effects Effects 0.000 description 4
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 239000000284 extract Substances 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 230000004931 aggregating effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000001186 cumulative effect Effects 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 238000001303 quality assessment method Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W16/00—Network planning, e.g. coverage or traffic planning tools; Network deployment, e.g. resource partitioning or cells structures
- H04W16/22—Traffic simulation tools or models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W24/00—Supervisory, monitoring or testing arrangements
- H04W24/02—Arrangements for optimising operational condition
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Evolutionary Computation (AREA)
- Computer Networks & Wireless Communication (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
The invention provides a communication optimization method based on federal learning, belonging to the technical field of federal machine learning, comprising the following steps: acquiring local gradient core characteristics uploaded by each client; the local gradient core characteristics refer to data extracted by preprocessing target data by each client, training a local model and performing singular value decomposition on the local model; performing sketch gradient compression and sketch gradient aggregation on the local gradient core characteristics, extracting the first k sparse parameter matrixes, executing a federal average aggregation algorithm on the first k sparse parameter matrixes to obtain a global model, and sending the global model to each client; and calculating the accumulated gradient error value and the gradient error compensation value of each client, adaptively adjusting the weight of the gradient of the local gradient core characteristic to be compensated and the weight of the local gradient core characteristic in the process of sketch gradient aggregation, and then iterating. The method can effectively compress the gradient of the model, reduce communication overhead generated in model training and improve the accuracy of model prediction.
Description
Technical Field
The invention relates to the technical field of federal machine learning, in particular to a communication optimization method based on federal learning.
Background
The existing federal learning generally uploads the intermediate parameters of model training to a parameter server to realize model aggregation, and two problems exist in the process: firstly, large-scale distributed training needs a large amount of communication bandwidth for gradient exchange, which limits the expandability of multi-node training; the second is the need for expensive high bandwidth network infrastructure.
When distributed training is performed on a mobile device, there is higher network latency, lower system throughput, and intermittent poor connectivity.
In order to solve the above problem, if the traditional SVD singular value decomposition method is directly adopted, although the accuracy of model prediction is improved, the gradient compression effect is not ideal. If the sketch compression method is adopted, although the gradient compression effect is ideal, the accuracy of model prediction is reduced.
Disclosure of Invention
The invention aims to provide a communication optimization method based on federal learning, which can effectively compress model gradient, reduce communication overhead generated in model training and improve the accuracy of model prediction.
In order to achieve the purpose, the invention adopts the technical scheme that:
a communication optimization method based on federal learning comprises the following steps: s1, acquiring local gradient core characteristics uploaded by each client; the local gradient core characteristics refer to model gradients extracted by preprocessing target data by each client to train a local model and performing singular value decomposition on the local model; the local model is a federal model; s2, carrying out sketch gradient compression on the local gradient core characteristics, carrying out sketch gradient aggregation, extracting the first k sparse parameter matrixes, executing a Federal average aggregation algorithm on the first k sparse parameter matrixes to obtain a global model, and distributing the global model to each client; s3, calculating the accumulated gradient error value and the gradient error compensation value of each client, adaptively adjusting the gradient of the local gradient core characteristic to be compensated and the weight of the local gradient core characteristic in the process of sketch gradient aggregation, and then iteratively executing S1 to S3.
Further, preprocessing the target data includes: dividing target data into a plurality of data sets, and executing uniform random sampling operation to select the batch size of the data sets in each iteration to obtain a target data set; performing paradigm boundary calculation on the target data set; acquiring initialized global model parameters, training a local model through a target data set after the normal form boundary calculation, and calculating the local gradient of the local model; and cutting the local gradient to obtain a gradient matrix of the local model.
Further, the local model is subjected to singular value decomposition as: and each client executes a rapid gradient descent algorithm, decomposes the gradient matrix of the local model into a left singular vector, a right singular vector and a diagonal matrix containing the gradient, and obtains the core characteristic of the local gradient.
Further, each client executes a fast gradient descent algorithm, decomposes the gradient matrix of the local model into a left singular vector, a right singular vector and a diagonal matrix containing gradients, and obtains local gradient core features, including: storing data of a gradient matrix of a local model in a zero matrix in sequence, and compressing until zero-value rows do not exist in the zero matrix to obtain a first matrix; constructing a sub-sampled random Hadamard transform matrix, calculating the product of the sub-sampled random Hadamard transform matrix and a first matrix, and compressing the first matrix from m toObtaining a second matrix; and executing an SVD algorithm on the second matrix, decomposing the gradient matrix of the local model into a left singular vector, a right singular vector and a diagonal matrix containing gradients, and obtaining the core characteristics of the local gradients.
Further, S3 includes: calculating a local model loss value of each client in the current iteration process; calculating the cross entropy of the loss value of the local model, and calculating the quality evaluation weight of the local model according to the cross entropy; calculating local gradient errors generated by draft gradient compression in the draft gradient aggregation process to obtain accumulated gradient error values and gradient error compensation values of all clients; adjusting the weight of the local gradient core characteristics of each client for sketch gradient aggregation according to the quality evaluation weight and the accumulated gradient error value of the local model of each client; and adaptively adjusting the gradient to be compensated of the local gradient core characteristic of the whole client according to the gradient error compensation value, and entering the next iteration.
Drawings
The invention and its features, aspects and advantages will become more apparent from reading the following detailed description of non-limiting embodiments with reference to the accompanying drawings. Like reference symbols in the various drawings indicate like elements. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the invention.
FIG. 1 is a general flow chart of a federated learning-based communication optimization method provided by the present invention;
FIG. 2 is a detailed flowchart of a preprocessing algorithm in a federated learning-based communication optimization method provided by the present invention;
FIG. 3 is a detailed flowchart of a fast gradient descent algorithm in a communication optimization method based on federated learning according to the present invention;
fig. 4 is a specific flowchart for performing gradient error compensation in the communication optimization method based on federal learning according to the present invention.
Detailed Description
The invention will be further described with reference to the following drawings and specific examples, which should not be construed as limiting the invention thereto.
The method aims to solve the problems that in the prior art, when the intermediate parameters of model training are uploaded to a parameter server to realize model aggregation in a Federal learning algorithm, a traditional SVD singular value decomposition method is directly adopted, although the accuracy of model prediction is improved, the gradient compression effect is not ideal; the method is characterized in that a sketch compression method is directly adopted, although the gradient compression effect is ideal, the problem of the accuracy of model prediction is reduced, therefore, as shown in fig. 1, the invention provides a communication optimization method based on federal learning, in particular to a mobile edge network communication optimization method, aiming at the problems that when distributed training is carried out under a mobile edge network, the resources of a mobile terminal are limited, and the training can bring higher network delay, lower system throughput and intermittent network connection failure, the communication optimization is carried out on the communication optimization, and the huge communication overhead generated in the model training process is reduced.
Firstly, each client needs to preprocess local target data, and then trains out a local model, wherein the local model is a federal model; as shown in fig. 2, the steps specifically include:
splitting target data into minipatch datasetsAnd executing uniform random sampling operation to select the batch size of the data set in each iteration to obtain a target data set d'i=di⊙Sc(ii) a Wherein i is the ith client, diIs the data of the ith client, c is the number of matrix columns, ScInputting a random uniform sampling matrix with the same dimension as the sensitive original data;
since each client is to be in target dataset d'iRun a Federal learning model to extract features, d'il=F(d′i) To limit a certain global sensitivity, the target data set d'iPerforming normal form boundary calculations, i.e.Wherein,is the minimum data batch size; dlAn infinite norm representing a first level output of the neural network model; d 'can be known from the formula'lHas an upper limit of Sf=max||f(d)-f(d′)||2(ii) a Wherein f is a real-valued function, provided thatdlThe sensitivity of the data can be protected, and the sensitive data can be protected from being leaked to a certain extent;
the global model parameters w are then initializedglobalAnd distributing the data to each client, training a local model by each client according to the target data set calculated by the paradigm boundary, and calculating the local gradient g of the local model(i)(ii) a Clipping local gradients, i.e.So as to prevent gradient explosion and finally obtain the gradient matrix A of the local model.
Then each client executes a rapid gradient descent algorithm, as shown in fig. 3, a gradient matrix of the local model is decomposed into a left singular vector, a right singular vector and a diagonal matrix containing gradients to approximately describe matrix information of a certain percentage of the gradient matrix, and then the gradients are compressed in a matrix form to obtain local gradient core characteristics, so that the high communication cost generated by federal training is reduced; the method specifically comprises the following steps:
collecting gradient matrix A epsilon R of local model in sequencem×nN data points in the mth row and stored in the zero matrix F e Rm×nCompressing all zero-value rows in the zero matrix to obtain a first matrix F';
when judging that zero-value rows do not exist in the F, constructing a sub-sampling random Hadamard transform matrix phi = N.H.D; wherein N ∈ Rq×mRepresents a scaled random matrix that is uniformly sampled per row without replacement from m rows of an m identity matrix, rescaled toD represents an m x m diagonal matrix with elements of i.i.d, being a random variable (equal probability-1 or 1); h belongs to { +/-1 }m×mExpressed as Hadamard matrix (Hadamard) meaning hij=(-1)<i-1,j-1>,<i-1,j-1>Is the dot product of the b-bit binary vectors of integers i-1 and j-1,the i, j represents the ith row and the jth column) Hadamard matrix can also be expressed asWhere m is the size of the matrix;
computing the product of the sub-sampled random Hadamard transform matrix Φ and the first matrix F 'to compress the size of F' from m toAnd connecting the new compressed gradient and the shrunk gradient to obtain a second matrix B epsilon Rk×n;
Performing SVD algorithm on the second matrix according to [ U, lambda, V]= SVD (B) decompose gradient matrix of local model into left singular vector, right singular vector and diagonal matrix containing gradient and passCompressing the original information to the front of BAnd improving the communication efficiency of uploading the local model parameters under the condition that the influence on the local gradient kernel is small.
Performing sketch gradient compression on local gradient core characteristics of each client, performing sketch gradient aggregation, extracting the first k sparse parameter matrixes, performing a federal average aggregation algorithm on the first k sparse parameter matrixes to obtain a global model, and distributing the global model to each client;
then, as shown in fig. 4, calculating an accumulated gradient error value and a gradient error compensation value of each client, adaptively adjusting the gradient of the local gradient core feature to be compensated and the weight of the local gradient core feature during sketch gradient aggregation, and substituting the weights into the next iterative computation global model; the method specifically comprises the following steps:
calculating the locality of each client in the current iteration processLoss value of modelWherein, DOiIs the ith client(s),is a parameter of the local model of the ith client in the t-th iteration;
calculating the cross entropy H (f (x) of the local model loss valuei),yi)=-∑xiyilogf(xi) Evaluating local model penalty values by cross entropyIf the cross entropy H (f (x)i),yi) The smaller the value is, the closer the local model prediction result is to the real result is, which indicates that the model training quality is higher;
then, the quality evaluation weight of the local model is calculated according to the cross entropyWherein (x)i,yi) Is the local data tag, x, of the ith clientiIs input data, yiIs the desired output, f (x)i) Is the local model prediction result of the ith client;
aggregating the quality assessment weights of the clients, client D, each time in a sketch gradient manneriThe local gradient error generated by the compression of the gradient of the sketch in the gradient aggregation process of the sketch can be used as the accumulated gradient error valueIt is shown that the process of the present invention,the accumulated gradient error value of the ith client in the t-th iteration is represented, beta is an error factor, and the accumulated gradient error value can be continuously updated along with the iteration; then according toWherein α is a compensation coefficient;is the gradient after SVD and Sketch;
adjusting the weight of the local gradient core characteristics of each client for sketch gradient aggregation according to the quality evaluation weight and the accumulated gradient error value of the local model of each client; according to the gradient error compensation value to adaptively adapt to the gradient to be compensated of the local gradient core characteristic of the whole client, optimal parameters alpha and beta are selected and substituted into the next iterative computation, so that the accuracy of the global model is improved, and the optimal parameters alpha and beta are expressed as the gradient to be compensated in the global model updating computation processWhere n is the number of clients participating in model training, ψ is the number of high quality clients selected uniformly and randomly, η is the local model learning rate, Pi(Di) Is a client DiThe quality evaluation weight of the local model in the ith model aggregation, wherein the model aggregation is to perform federal averaging on uploaded gradient updates of all clients and is used for updating the global model);is the local gradient that client i did not compensate in the t-th iteration, alpha is the compensation coefficient,is the cumulative gradient error value of the ith client in the tth iteration.
In summary, according to the communication optimization method based on federal learning provided by the present invention, each client independently trains a local model, then performs Singular Value Decomposition (SVD) on the local model, extracts local gradient core features, sends the local gradient core features to a parameter server, performs Sketch (Sketch) gradient compression by the parameter server, then performs Sketch (Sketch) gradient aggregation, extracts the first k approximate gradient values from the aggregated parameters, performs the federal average aggregation algorithm on the first k approximate gradient values to obtain a global model, and sends the global model to each client to perform the next iteration process. And aiming at local gradient errors generated after the Sketch gradient lossy compression, self-adaptive gradient error accumulation and gradient error compensation are supported, the gradient to be compensated is self-adaptively adjusted, the gradient is added into the next iteration, the model gradient can be effectively compressed through SVD and Sketch compression, the communication overhead generated in model training is reduced, the gradient errors between SVD and Sketch operation and SVD operation are only executed due to the combination of SVD and Sketch operation are solved through the gradient error compensation, and therefore the accuracy of global model prediction is improved. The invention can greatly reduce the communication overhead between the client and the server in the whole training process, accelerate the convergence of the model and ensure that the accuracy of the model prediction is almost the same as the original prediction accuracy.
The above description is of the preferred embodiment of the invention; it is to be understood that the invention is not limited to the particular embodiments described above, in that devices and structures not described in detail are understood to be implemented in a manner common in the art; any person skilled in the art can make many possible variations and modifications, or modify equivalent embodiments, without departing from the technical solution of the invention, without affecting the essence of the invention; therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention are within the scope of the technical solution of the present invention, unless the technical essence of the present invention is not departed from the content of the technical solution of the present invention.
Claims (5)
1. A communication optimization method based on federal learning is characterized by comprising the following steps:
s1, acquiring local gradient core characteristics uploaded by each client; the local gradient core characteristics refer to model gradients extracted by preprocessing target data by each client, training a local model and performing singular value decomposition on the local model; the local model is a federal model;
s2, performing sketch gradient compression on the local gradient core features, performing sketch gradient aggregation, extracting the first k sparse parameter matrixes, performing a federal average aggregation algorithm on the first k sparse parameter matrixes to obtain a global model, and distributing the global model to each client;
s3, calculating accumulated gradient error values and gradient error compensation values of all the clients, adaptively adjusting the gradient of the local gradient core characteristics to be compensated and the weight of the local gradient core characteristics during sketch gradient aggregation, and then iteratively executing S1 to S3.
2. The method of claim 1, wherein the preprocessing the target data comprises:
dividing target data into a plurality of data sets, and executing uniform random sampling operation to select the batch size of the data sets in each iteration to obtain a target data set;
performing paradigm boundary calculation on the target data set;
acquiring initialized global model parameters, training a local model through a target data set after the normal form boundary calculation, and calculating the local gradient of the local model;
and cutting the local gradient to obtain a gradient matrix of the local model.
3. The federal learning-based communication optimization method as claimed in claim 2, wherein the singular value decomposition of the local model is as follows:
and each client executes a rapid gradient descent algorithm, and decomposes the gradient matrix of the local model into a left singular vector, a right singular vector and a diagonal matrix containing gradients to obtain local gradient core characteristics.
4. The federal learning-based communication optimization method of claim 3, wherein each client executes a fast gradient descent algorithm to decompose a gradient matrix of the local model into a left singular vector, a right singular vector and a diagonal matrix containing gradients, so as to obtain local gradient core features, and the method includes:
storing data of a gradient matrix of a local model in a zero matrix in sequence, and compressing until zero-value rows do not exist in the zero matrix to obtain a first matrix;
constructing a sub-sampled random Hadamard transform matrix, calculating the product of the sub-sampled random Hadamard transform matrix and a first matrix, compressing the first matrix from m toObtaining a second matrix;
and executing an SVD algorithm on the second matrix, decomposing the gradient matrix of the local model into a left singular vector, a right singular vector and a diagonal matrix containing gradients, and obtaining the core characteristics of the local gradients.
5. The method according to claim 4, wherein the S3 comprises:
calculating a local model loss value of each client in the current iteration process;
calculating the cross entropy of the loss value of the local model, and calculating the quality evaluation weight of the local model according to the cross entropy;
calculating local gradient errors generated by the gradient compression of the sketch in the gradient aggregation process of the sketch to obtain accumulated gradient error values and gradient error compensation values of all clients;
adjusting the weight of the local gradient core characteristics of each client for sketch gradient aggregation according to the quality evaluation weight and the accumulated gradient error value of the local model of each client; and adaptively adjusting the gradient to be compensated of the local gradient core characteristic of the whole client according to the gradient error compensation value, and entering the next iteration.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210906790.4A CN115278709B (en) | 2022-07-29 | 2022-07-29 | Communication optimization method based on federal learning |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210906790.4A CN115278709B (en) | 2022-07-29 | 2022-07-29 | Communication optimization method based on federal learning |
Publications (2)
Publication Number | Publication Date |
---|---|
CN115278709A true CN115278709A (en) | 2022-11-01 |
CN115278709B CN115278709B (en) | 2024-04-26 |
Family
ID=83772330
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202210906790.4A Active CN115278709B (en) | 2022-07-29 | 2022-07-29 | Communication optimization method based on federal learning |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN115278709B (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117216596A (en) * | 2023-08-16 | 2023-12-12 | 中国人民解放军总医院 | Federal learning optimization communication method, system and storage medium based on gradient clustering |
CN117692939A (en) * | 2024-02-02 | 2024-03-12 | 南京邮电大学 | Client scheduling method in dynamic communication environment |
Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20190087713A1 (en) * | 2017-09-21 | 2019-03-21 | Qualcomm Incorporated | Compression of sparse deep convolutional network weights |
US20200117580A1 (en) * | 2018-04-22 | 2020-04-16 | Sas Institute Inc. | Validation Sets for Machine Learning Algorithms |
US20200142948A1 (en) * | 2018-11-07 | 2020-05-07 | Samsung Electronics Co., Ltd. | System and method for cached convolution calculation |
CN111507481A (en) * | 2020-04-17 | 2020-08-07 | 腾讯科技(深圳)有限公司 | Federated learning system |
CN112199702A (en) * | 2020-10-16 | 2021-01-08 | 鹏城实验室 | Privacy protection method, storage medium and system based on federal learning |
CN112288097A (en) * | 2020-10-29 | 2021-01-29 | 平安科技(深圳)有限公司 | Federal learning data processing method and device, computer equipment and storage medium |
CN112862011A (en) * | 2021-03-31 | 2021-05-28 | 中国工商银行股份有限公司 | Model training method and device based on federal learning and federal learning system |
CN113222179A (en) * | 2021-03-18 | 2021-08-06 | 北京邮电大学 | Federal learning model compression method based on model sparsification and weight quantization |
CN113645197A (en) * | 2021-07-20 | 2021-11-12 | 华中科技大学 | Decentralized federal learning method, device and system |
CN114339252A (en) * | 2021-12-31 | 2022-04-12 | 深圳大学 | Data compression method and device |
-
2022
- 2022-07-29 CN CN202210906790.4A patent/CN115278709B/en active Active
Patent Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20190087713A1 (en) * | 2017-09-21 | 2019-03-21 | Qualcomm Incorporated | Compression of sparse deep convolutional network weights |
US20200117580A1 (en) * | 2018-04-22 | 2020-04-16 | Sas Institute Inc. | Validation Sets for Machine Learning Algorithms |
US20200142948A1 (en) * | 2018-11-07 | 2020-05-07 | Samsung Electronics Co., Ltd. | System and method for cached convolution calculation |
CN111507481A (en) * | 2020-04-17 | 2020-08-07 | 腾讯科技(深圳)有限公司 | Federated learning system |
CN112199702A (en) * | 2020-10-16 | 2021-01-08 | 鹏城实验室 | Privacy protection method, storage medium and system based on federal learning |
CN112288097A (en) * | 2020-10-29 | 2021-01-29 | 平安科技(深圳)有限公司 | Federal learning data processing method and device, computer equipment and storage medium |
CN113222179A (en) * | 2021-03-18 | 2021-08-06 | 北京邮电大学 | Federal learning model compression method based on model sparsification and weight quantization |
CN112862011A (en) * | 2021-03-31 | 2021-05-28 | 中国工商银行股份有限公司 | Model training method and device based on federal learning and federal learning system |
CN113645197A (en) * | 2021-07-20 | 2021-11-12 | 华中科技大学 | Decentralized federal learning method, device and system |
CN114339252A (en) * | 2021-12-31 | 2022-04-12 | 深圳大学 | Data compression method and device |
Non-Patent Citations (3)
Title |
---|
LAIZHONG CUI;XIAOXIN SU;YIPENG ZHOU;LEI ZHANG: ""ClusterGrad: Adaptive Gradient Compression by Clustering in Federated Learning"", 《GLOBECOM 2020 - 2020 IEEE GLOBAL COMMUNICATIONS CONFERENCE》, 25 January 2021 (2021-01-25) * |
P. RAVICHANDRAN;C. SARAVANAKUMAR;J. DAFNI ROSE;M. VIJAYAKUMAR;V. MUTHU LAKSHMI: ""Efficient Multilevel Federated Compressed Reinforcement Learning of Smart Homes Using Deep Learning Methods"", 《2021 INTERNATIONAL CONFERENCE ON INNOVATIVE COMPUTING, INTELLIGENT COMMUNICATION AND SMART ELECTRICAL SYSTEMS (ICSES)》, 16 December 2021 (2021-12-16) * |
杨阳: "物联网中数据压缩与资源分配的研究", 《中国博士学位论文全文数据库(电子期刊)》, 15 May 2020 (2020-05-15) * |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117216596A (en) * | 2023-08-16 | 2023-12-12 | 中国人民解放军总医院 | Federal learning optimization communication method, system and storage medium based on gradient clustering |
CN117216596B (en) * | 2023-08-16 | 2024-04-30 | 中国人民解放军总医院 | Federal learning optimization communication method, system and storage medium based on gradient clustering |
CN117692939A (en) * | 2024-02-02 | 2024-03-12 | 南京邮电大学 | Client scheduling method in dynamic communication environment |
CN117692939B (en) * | 2024-02-02 | 2024-04-12 | 南京邮电大学 | Client scheduling method in dynamic communication environment |
Also Published As
Publication number | Publication date |
---|---|
CN115278709B (en) | 2024-04-26 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN115278709A (en) | Communication optimization method based on federal learning | |
CN111555781B (en) | Large-scale MIMO channel state information compression and reconstruction method based on deep learning attention mechanism | |
CN111898703B (en) | Multi-label video classification method, model training method, device and medium | |
CN111259917B (en) | Image feature extraction method based on local neighbor component analysis | |
CN114332500B (en) | Image processing model training method, device, computer equipment and storage medium | |
CN115829027A (en) | Comparative learning-based federated learning sparse training method and system | |
CN115587633A (en) | Personalized federal learning method based on parameter layering | |
CN113658130A (en) | No-reference screen content image quality evaluation method based on dual twin network | |
CN116168197A (en) | Image segmentation method based on Transformer segmentation network and regularization training | |
CN115865145A (en) | Large-scale MIMO channel state information feedback method based on Transformer | |
CN114626550A (en) | Distributed model collaborative training method and system | |
CN117236201B (en) | Diffusion and ViT-based downscaling method | |
CN116797850A (en) | Class increment image classification method based on knowledge distillation and consistency regularization | |
CN114677545B (en) | Lightweight image classification method based on similarity pruning and efficient module | |
CN115952860A (en) | Heterogeneous statistics-oriented clustering federal learning method | |
CN115909441A (en) | Face recognition model establishing method, face recognition method and electronic equipment | |
CN114677535A (en) | Training method of domain-adaptive image classification network, image classification method and device | |
CN112116062A (en) | Multilayer perceptron nonlinear compression method based on tensor string decomposition | |
Limmer et al. | Optimal deep neural networks for sparse recovery via Laplace techniques | |
CN117222005B (en) | Fingerprint positioning method, fingerprint positioning device, electronic equipment and storage medium | |
CN110111326B (en) | Reconstructed image quality evaluation method based on ERT system | |
CN115019079B (en) | Method for accelerating deep learning training by distributed outline optimization for image recognition | |
CN114581946B (en) | Crowd counting method and device, storage medium and electronic equipment | |
CN112598130B (en) | Soil moisture data reconstruction method based on self-encoder and singular value threshold and computer readable storage medium | |
Li et al. | HyperFeel: An Efficient Federated Learning Framework Using Hyperdimensional Computing |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |