CN113516249A - Federal learning method, system, server and medium based on semi-asynchronization - Google Patents

Federal learning method, system, server and medium based on semi-asynchronization Download PDF

Info

Publication number
CN113516249A
CN113516249A CN202110681063.8A CN202110681063A CN113516249A CN 113516249 A CN113516249 A CN 113516249A CN 202110681063 A CN202110681063 A CN 202110681063A CN 113516249 A CN113516249 A CN 113516249A
Authority
CN
China
Prior art keywords
model
group
client
version number
local
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110681063.8A
Other languages
Chinese (zh)
Other versions
CN113516249B (en
Inventor
刘铎
张宇
段莫名
李丽
陈咸彰
任骜
谭玉娟
汪成亮
梁靓
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chongqing University
Original Assignee
Chongqing University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chongqing University filed Critical Chongqing University
Priority to CN202110681063.8A priority Critical patent/CN113516249B/en
Publication of CN113516249A publication Critical patent/CN113516249A/en
Application granted granted Critical
Publication of CN113516249B publication Critical patent/CN113516249B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Medical Informatics (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Information Transfer Between Computers (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a federal learning method based on semi-asynchronization, which groups client ends on a central server according to the local model update direction and update time of each client end to obtain a plurality of client end groups, creates a plurality of scheduling processes taking the client end groups as units so that the scheduling processes initialize the group model version number of the group in each communication wheel, calculates the difference between the local model version number of the client end and the group model version number in each communication wheel to be used for measuring the outdated degree of the local model of the client end version number, further enables the client end to select a synchronous update or asynchronous update mode to update the local model according to the comparison result of the difference of the version numbers and a preset model outdated threshold value, can effectively solve the dequeue effect of the client end and can solve the model outdated problem of the client end, thereby improving the accuracy of the model.

Description

Federal learning method, system, server and medium based on semi-asynchronization
Technical Field
The invention relates to the technical field of machine learning, in particular to a federal learning method and a system based on semi-asynchronization, a central server and a computer readable storage medium.
Background
Federal learning is an emerging machine learning framework, and aims to help a plurality of organizations utilize data and perform machine learning modeling under the condition of meeting the requirements of user privacy, data safety and government laws and regulations. In the federal learning system, a client is a mobile or internet of things device which can be connected, and a server is a cloud server. The cloud server plays a role in maintaining and deploying a global machine learning model, and each client downloads the global model on the cloud server as a local model. The action mechanism of the system is composed of a plurality of communication wheels, in each communication wheel, a cloud server deploys a global model to each client, each client trains a local model by taking a stored data set of the client as a training data set, and the cloud server collects the models and integrates the models into a new global model. The number of communication rounds is determined by the specific training task. In the whole training process of the system, the transmitted data is the model, but not the training data on the client, so that the data safety is guaranteed.
The existing federal learning system mainly has two model updating mechanisms:
the synchronous updating mechanism comprises the following steps: in the synchronous federal learning system, all clients need to download the global model on the cloud server on a uniform time node, and the cloud server needs to wait for all clients to complete training tasks. The synchronous federated learning has the advantages of high model convergence speed and high precision.
An asynchronous update mechanism: in the asynchronous federated learning system, the central server only needs to integrate the model parameters transmitted by the clients which have completed the training task, and does not need to wait for all the clients, and each client can request a new global model from the server after the training process is completed. This update mechanism prevents the server from entering an empty state. For example, an asynchronous online federated learning system in which clients learn online using a continuous stream of local data and update global models in an asynchronous manner.
In the implementation of the above process, the inventor finds that in the above federate learning system based on the synchronous update mechanism, there are some client-side dequeues, and the central server falls into an empty state. The queue-dropping effect means that some clients cannot complete the training task or train slowly due to insufficient network resources or hardware resources.
While the asynchronous update mechanism has natural advantages for solving the queue-dropping effect, the update mechanism has some disadvantages. On the one hand, this mechanism makes the data transfer large, possibly leading to a server crash. On the other hand, this mechanism has the problem that the model is outdated. That is, when the server receives a local model uploaded by a client with a slower update, the global model on the server has been updated many times by other clients, the local model of the client has become obsolete, and the obsolete model may degrade the performance of the model. In addition, there is a model delay between models of each client and server, and the model delay of each client is unstable due to the influence of network environment, device environment, and the like. Due to the instability of model delay, the accuracy of asynchronous updating is greatly reduced.
Disclosure of Invention
Aspects of the present invention provide a federated learning method, system, central server, and computer-readable storage medium based on semi-asynchronization, which can solve the problem of the fall-behind effect in the existing synchronous federated learning, and the problems of model obsolescence and low model precision in the asynchronous federated learning.
The federal learning method based on semi-asynchronism provided by the first aspect of the invention comprises the following steps:
the central server broadcasts the global model parameters to all the clients;
each client receives the global model parameters, trains according to the global model parameters and local data, obtains the updating direction of the local model of the client and the updating time of the local model, and sends the updating direction of the local model and the updating time of the local model to the central server;
the central server groups each client according to the received updating direction of the local model of each client and the received updating time of the local model to obtain a grouped client group, and creates a plurality of processes taking the client group as a unit to obtain a plurality of scheduling processes; or, the central server groups each client according to the received update direction of the local model of each client and the update time of the local model, so as to obtain a grouped client group, and sends a grouping deployment request to each edge computing device, so that each edge computing device responds to the grouping deployment request to create a scheduling process of the group to which the edge computing device belongs; each scheduling process is used for deploying a group model of a client group, initializing a group model version number of the client group when each communication round starts, and sending group model parameters, the group model version number, a preset model outdated threshold value and a preset total time budget of each round to a client of the group in each communication round;
for each client, the client initializes the version number of a local model of the client in the current communication wheel, receives the group model version number, the preset model outdated threshold and the preset total time budget of each wheel, which are sent by the scheduling process of the group, and updates the version number of the local model according to the group model version number when the group model version number sent by the scheduling process is detected at any moment of the current communication wheel;
each client selects a corresponding updating mode to update the parameters and the version number of the local model according to the difference between the group model parameters, the preset model outdated threshold value, the version number of the local model and the group model version number in the time of the preset total time budget of each communication wheel, and sends the updated parameters and the updated version number of the local model to the scheduling process of the group, so that the scheduling process of the group updates the group model and the group model version number of the group, and sends the updated group model and the updated group model version number to the clients of the group; the updating mode comprises synchronous updating and asynchronous updating.
Preferably, each of the clients selects a corresponding update mode to update the parameter and the version number of the local model according to the group model parameter, the preset model outdated threshold, and the difference between the version number of the local model and the version number of the group model in the time of the preset total time budget of each communication round, and sends the updated parameter and the updated version number of the local model to the scheduling process of the belonging group, so that the scheduling process of the belonging group updates the group model and the group model version number of the belonging group, and sends the updated group model and the updated group model version number to the clients of the belonging group, which specifically includes:
for each client, when the client detects that the difference between the version number of the local model and the group model version number of the affiliated group is smaller than the preset model outdated threshold value within the preset total time budget of each round of the current communication round, the client performs local training according to local data and received group model parameters of the affiliated group to obtain updated parameters and version numbers of the local model, and asynchronously sends the updated parameters and version numbers of the local model of the client to the scheduling process of the affiliated group, so that the scheduling process of the affiliated group updates the group model and the group model version number of the affiliated group, and sends the updated group model and the updated group model version number to all clients of the affiliated group requesting group model information;
when the client detects that the difference between the version number of the local model and the version number of the group model of the belonging group is larger than the preset model passing threshold value in the current communication wheel, sending a synchronous updating request to the scheduling process of the belonging group, so that the scheduling process of the belonging group responds to the synchronous updating request and sends a synchronous updating command to all the clients of the belonging group with time budget;
and all the clients receiving the synchronous updating command execute local training operation, synchronously update the parameters and the version numbers of the local models, send the updated version numbers of the local models and the updated local model parameters to the scheduling process of the affiliated group, so that the scheduling process of the affiliated group updates the group models and the group model version numbers of the affiliated group, and send the updated group models and the updated group model version numbers to all the clients of the affiliated group with the time budget.
Preferably, the central server groups each client according to the received update direction of the local model of each client and the received update time of the local model, to obtain a grouped client group, and specifically includes:
and the central server groups each client by adopting a clustering algorithm according to the received updating direction of the local model of each client and the received updating time of the local model, so as to obtain a grouped client group.
The second aspect of the present invention provides a federated learning method based on semi-asynchronization, which is applied to a client and includes:
receiving the global model parameters broadcasted by the central server, training according to the global model parameters and local data to obtain the updating direction of the local model of the client and the updating time of the local model, and sending the updating direction of the local model and the updating time of the local model to the central server so that the central server executes the operation of grouping the clients and creating a scheduling process; wherein the client grouping operation is to: the central server groups each client according to the received updating direction of the local model of each client and the received updating time of the local model, so as to obtain a grouped client group; the operation of the central server for establishing the scheduling process is as follows: creating a plurality of processes taking a client group as a unit according to the grouping result of the clients to obtain a plurality of scheduling processes; each scheduling process is used for deploying a group model of a client group, initializing a group model version number, a preset model outdated threshold value and a preset total time budget of each communication round of the client group when each communication round starts, and sending a group model parameter, the group model version number, the preset model outdated threshold value and the preset total time budget of each round to all the clients of the group in each communication round;
initializing the version number of a local model of the client when the current communication wheel starts, receiving the group model version number, the preset model outdated threshold and the preset total time budget of each wheel sent by the scheduling process of the belonging group, and updating the version number of the local model according to the group model version number when the group model version number sent by the scheduling process is detected at any moment of the current communication wheel;
selecting a corresponding updating mode to update the parameters and the version number of the local model according to the group model parameters, the preset model outdated threshold value, the difference between the version number of the local model and the group model version number in the time of the preset total time budget of each communication wheel, and sending the updated parameters and the updated version number of the local model to the scheduling process of the belonging group, so that the scheduling process of the belonging group updates the group model and the group model version number of the belonging group, and sends the updated group model and the updated group model version number to the client of the belonging group; the updating mode comprises synchronous updating and asynchronous updating.
The invention provides a federated learning method based on semi-asynchronization in a third aspect, which is applied to a central server, and comprises the following steps:
broadcasting the global model parameters to all the clients so that each client receives the global model parameters, training according to the global model parameters and local data to obtain the updating direction of the local model of the client and the updating time of the local model, and sending the updating direction of the local model and the updating time of the local model to the central server;
grouping each client according to the received updating direction of the local model of each client and the received updating time of the local model to obtain a grouped client group, and creating a plurality of processes taking the client group as a unit to obtain a plurality of scheduling processes; each scheduling process is used for deploying a group model of a client group, initializing a group model version number of the client group when each communication round starts, and sending group model parameters, the group model version number, a preset model outdated threshold value and a preset total time budget per round to all the clients of the group in each communication round so as to enable the clients to execute local training operation;
in the current communication wheel, receiving a local training result returned by the client, so that each scheduling process updates the group model and the group model version number of the group to which the scheduling process belongs according to the local training result, and sends the updated group model and the updated group model version number to the client of the group to which the scheduling process belongs;
wherein the result of the local training is obtained by:
the client initializes the version number of a local model of the client when a current communication wheel starts, receives the group model version number, the preset model outdated threshold and the preset total time budget of each wheel sent by a scheduling process of a group to which the client belongs, and updates the version number of the local model according to the group model version number when the group model version number sent by the scheduling process is detected at any moment of the current communication wheel;
the client selects a corresponding updating mode to update the parameters and the version number of the local model according to the group model parameters, the preset model outdated threshold, the difference between the version number of the local model and the group model version number within the time of the preset total time budget of each communication round to obtain the updated parameters and version number of the local model; the updating mode comprises synchronous updating and asynchronous updating.
The invention provides a federal learning system based on semi-asynchronization, which comprises a central server, a plurality of edge computing devices and a plurality of clients, wherein the central server, the clients and the edge computing devices are in communication connection;
the central server is used for broadcasting the global model parameters to all the clients;
the client is used for receiving the global model parameters, training according to the global model parameters and local data to obtain the updating direction of the local model of the client and the updating time of the local model, and sending the updating direction of the local model and the updating time of the local model to the central server;
the central server is further configured to group each client according to the received update direction of the local model of each client and the received update time of the local model, obtain a grouped client group, send a grouping result to each client, and send a grouping deployment request to each edge computing device; wherein each edge computing device manages a client group;
the edge computing device is used for responding to a grouping deployment request of the central server, deploying a group model of a client group, initializing a group model version number of the client group at the beginning of each communication round, and sending group model parameters, the group model version number, a preset model outdating threshold value and a preset total time budget per round to all the clients of the group to which the client belongs in each communication round;
the client is further configured to initialize a version number of a local model of the client at the beginning of each communication round, receive the group model version number, the preset model obsolescence threshold value, and the preset total time budget for each round sent by the scheduling process of the group to which the client belongs, and update the version number of the local model according to the group model version number when the group model version number sent by the scheduling process is detected at any time of each communication round; and the edge computing device is used for selecting a corresponding updating mode according to the group model parameter, the preset model outdated threshold value, the difference between the version number of the local model and the group model version number of the group to which the local model belongs and sending the updated parameter and version number of the local model to the edge computing device of the group to which the local model belongs within the preset total time budget of each communication wheel; the updating mode comprises synchronous updating and asynchronous updating.
The edge computing device is further configured to update the group model and the group model version number of the belonging group in each communication round according to the received updated local model parameter and version number of each client of the belonging group, and send the updated group model and the updated group model version number to the clients of the belonging group.
A fifth aspect of the present invention provides a computer-readable storage medium comprising a stored computer program, wherein when the computer program runs, a device on which the computer-readable storage medium is located is controlled to execute the method for federated learning based on semi-asynchrony as provided in the second aspect or the third aspect.
Compared with the prior art, the invention has the beneficial effects that: the invention provides a federated learning method based on semi-asynchronization, which groups the clients on a central server according to the local model update direction and update time of each client to obtain a plurality of client groups, creates a plurality of scheduling processes taking the client groups as units, so that the scheduling processes initialize the group model version number of the group in each communication wheel, calculates the difference between the local model version number of the client and the group model version number in each communication wheel to be used for measuring the outdated degree of the local model of the client version number, further enables the client to select a synchronous update or asynchronous update mode to update the local model according to the comparison result of the difference of the version numbers and a preset model outdated threshold value, can effectively solve the dequeue effect of the client and can solve the model outdated problem of the client, thereby improving the accuracy of the model. Correspondingly, the invention also provides a federal learning system based on semi-asynchronization, a central server and a computer readable storage medium.
Drawings
FIG. 1 is a flow chart of a federated learning method based on semi-asynchronization according to an embodiment of the present invention;
FIG. 2 is a flowchart of client grouping in a federated learning method based on semi-asynchronization according to an embodiment of the present invention;
fig. 3 is a flowchart of client training in the federated learning method based on semi-asynchronization according to an embodiment of the present invention;
FIG. 4 is a block diagram of a system architecture of a federated learning method based on semi-asynchronization according to an embodiment of the present invention;
fig. 5 is a framework diagram of the federal learning system based on semi-asynchrony according to the fourth embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, which is a flowchart of a federated learning method based on semi-asynchronization according to an embodiment of the present invention,
the federal learning method based on semi-asynchronization provided by the first embodiment of the invention comprises the following steps S11 to S15:
step S11, the central server broadcasts the global model parameters to all the clients;
step S12, each client receives the global model parameters, trains according to the global model parameters and the local data, obtains the update direction of the local model of the client and the update time of the local model, and sends the update direction of the local model and the update time of the local model to the central server;
step S13, the central server groups each client according to the received update direction of the local model of each client and the update time of the local model to obtain a grouped client group, and creates a plurality of processes with the client group as a unit to obtain a plurality of scheduling processes; each scheduling process is used for deploying a group model of a client group, initializing a group model version number of the client group at the beginning of each communication round, and sending group model parameters, the group model version number, the preset model outdating threshold and the preset total time budget per round to all the clients of the group in each communication round;
step S14, for each client, the client initializes the version number of the local model of the client in the current communication round, receives the group model version number sent by the scheduling process of the group, the preset model outdated threshold and the preset total time budget per round, and updates the version number of the local model according to the group model version number when detecting the group model version number sent by the scheduling process at any moment of the current communication round;
step S15, each client selects a corresponding updating mode to update the parameter and the version number of the local model according to the group model parameter, the preset model outdated threshold value, the difference between the version number of the local model and the version number of the group model in the time of the preset total time budget of each communication wheel, and sends the updated parameter and the updated version number of the local model to the scheduling process of the belonging group, so that the scheduling process of the belonging group updates the group model and the group model version number of the belonging group, and sends the updated group model and the updated group model version number of the belonging group to the client of the belonging group; the updating mode comprises synchronous updating and asynchronous updating.
In the embodiment of the present invention, the central server groups the clients before training, the grouping operation only needs to be performed once, and then the client group performs distributed training according to the grouping result of step S13 in each training round.
In specific implementation, in order to obtain a better-performing model, multiple rounds of training procedures need to be performed, that is, the above steps S14 and S15 need to be performed multiple times until a preset communication round or a preset accuracy is reached. After a plurality of rounds of training are executed, the group models of all the scheduling processes can be integrated to obtain a new whole model.
As an alternative embodiment, in step S13, "the central server groups each of the clients according to the received update direction of the local model and the received update time of the local model of each of the clients, to obtain a grouped client group, and creates a plurality of processes in units of client groups, to obtain a plurality of scheduling processes" may be replaced by:
and the central server groups each client according to the received updating direction of the local model of each client and the received updating time of the local model to obtain a grouped client group, and sends a grouping deployment request to each edge computing device, so that each edge computing device responds to the grouping deployment request to create a scheduling process of the group to which the edge computing device belongs. Specifically, each edge computing device creates a scheduling process for managing the operation of local training of its corresponding client group.
That is, in the embodiment of the present invention, the application may be deployed on any device with communication and computing capabilities, such as a central server, a cloud server, a mobile device, an edge computing device, and the like, and is not necessarily limited to the process created by the central server. In an embodiment of the present invention, the preset model obsolescence threshold and the preset total time budget for each round are set by a central server, and then the values of the preset model obsolescence threshold and the preset total time budget for each round in each round of training are kept unchanged. It will be appreciated that when the scheduling process is created by an edge computing device in communication with a central server, in particular implementation, the central server transmits the preset model obsolete threshold and the preset total time per round budget to the scheduling process created by the edge computing device, such that the scheduling process transmits the preset model obsolete threshold and the preset total time per round budget to clients of the group to which the scheduling process belongs.
In an alternative embodiment, the "updating manner of the local model" in the step S12 may be obtained by:
wk←w0,
Figure BDA0003122518700000111
wherein, w0Is a global model parameter, w, initiated by the central serverkRepresenting local model parameters of client k, and dividing the data set on client k into batch data set with batch size B
Figure BDA0003122518700000112
b represents the selected batch of data,
Figure BDA0003122518700000113
η is a learning rate used for training the local model, and is set according to a specific learning task, and the commonly set value is 0.1 and 0.01; l is a loss function, optionally a squared error function or a negative log-likelihood function,
Figure BDA0003122518700000114
representing the loss functions L vs w as differential symbolskAnd (6) taking a derivative.
Referring to fig. 2, fig. 2 is a flowchart of client grouping in the federated learning method based on semi-asynchronization according to an embodiment of the present invention. In this embodiment of the present invention, clustering algorithm may be adopted for grouping, that is, in an optional implementation manner, in step 12, "the central server groups each of the clients according to the received update direction of the local model of each of the clients and the received update time of the local model, to obtain a grouped client group," specifically including:
and the central server groups each client by adopting a clustering algorithm according to the received updating direction of the local model of each client and the received updating time of the local model, so as to obtain a grouped client group.
In specific implementation, the clustering algorithm may select spectral clustering, k-means clustering algorithm, and the like according to characteristics of the data type, the data size, and the like of the data set. For example, for a data set with sparse features, a spectral clustering algorithm is adopted. The similarity index used for clustering can optionally select cosine similarity, Euclidean distance and the like.
Referring to fig. 3, fig. 3 is a flowchart of client training in a federated learning method based on semi-asynchronization according to an embodiment of the present invention. In an optional implementation manner of the embodiment of the present invention, in step S15, "each of the clients selects a corresponding update method to update the parameter and the version number of the local model according to the group model parameter, the preset model obsolete threshold, and the difference between the version number of the local model and the group model version number in the time of the preset total time budget of each communication round of the current communication round, and sends the updated parameter and the updated version number of the local model to the scheduling process of the belonging group, so that the scheduling process of the belonging group updates the group model and the group model version number of the belonging group, and sends the updated group model and the updated group model version number to the client of the belonging group," specifically includes:
for each client, when the client detects that the difference between the version number of the local model and the group model version number of the affiliated group is smaller than the preset model outdated threshold value within the preset total time budget of each round of the current communication round, the client performs local training according to local data and received group model parameters of the affiliated group to obtain updated parameters and version numbers of the local model, and asynchronously sends the updated parameters and version numbers of the local model of the client to the scheduling process of the affiliated group, so that the scheduling process of the affiliated group updates the group model and the group model version number of the affiliated group, and sends the updated group model and the updated group model version number to all clients of the affiliated group requesting group model information;
when the client detects that the difference between the version number of the local model and the version number of the group model of the belonging group is larger than the preset model passing threshold value in the current communication wheel, sending a synchronous updating request to the scheduling process of the belonging group, so that the scheduling process of the belonging group responds to the synchronous updating request and sends a synchronous updating command to all the clients of the belonging group with time budget;
and all the clients receiving the synchronous updating command execute local training operation, synchronously update the parameters and the version numbers of the local models, send the updated version numbers of the local models and the updated local model parameters to the scheduling process of the affiliated group, so that the scheduling process of the affiliated group updates the group models and the group model version numbers of the affiliated group, and send the updated group models and the updated group model version numbers to all the clients of the affiliated group with the time budget.
In the embodiment of the present invention, the client requesting group model information refers to a client that completes asynchronous update in the current communication round and requests the group model and the group model version number to the scheduling process, and the clients still having time budgets are measured according to the preset total time budget for each round, specifically, the client having a time period from the start of local training to a certain time point in the process of executing local training in one communication round that is less than the preset total time budget for each round.
Referring to fig. 4, fig. 4 is a system structure block diagram of a semi-asynchronous federal learning method according to an embodiment of the present invention, which illustrates a training process, where a quadrangle, a triangle, a star, a pentagon, etc. in a group are clients of the group. The downward solid arrow indicates that the scheduling process asynchronously distributes the group model parameters, the upward solid arrow indicates that the client asynchronously sends the local model parameters to the scheduling process, the downward dotted arrow indicates that the scheduling process synchronously distributes the group model parameters, and the upward dotted arrow indicates that the client synchronously sends the local model parameters to the scheduling process. In the embodiment of the invention, the clients are grouped through the step S13, and the clients with the model updating time similar to the model updating time are grouped into one group, so that the model delay is reduced, and the queue-dropping effect is relieved. The group model is deployed to the scheduling process through step S13, and after the client belonging to the group completes the training task, the model parameters are sent to the scheduling process, and the scheduling process integrates and distributes a new group model. Through the step S15, the client executes the synchronous and asynchronous training tasks alternately, which not only alleviates the impact of the dequeue effect of the synchronous update mechanism, but also alleviates the problems of model delay and server overload in the asynchronous update mechanism.
Example two
The federated learning method based on semi-asynchronization provided by the second embodiment of the present invention is applied to the client, and includes steps S21 to S23:
step S21, receiving the global model parameter broadcasted by the central server, training according to the global model parameter and the local data, obtaining the update direction of the local model of the client and the update time of the local model, and sending the update direction of the local model and the update time of the local model to the central server, so that the central server executes the operation of grouping the clients and creating the scheduling process; wherein the client grouping operation is to: the central server groups each client according to the received updating direction of the local model of each client and the received updating time of the local model, so as to obtain a grouped client group; the operation of the central server for establishing the scheduling process is as follows: creating a plurality of processes taking a client group as a unit according to the grouping result of the clients to obtain a plurality of scheduling processes; each scheduling process is used for deploying a group model of a client group, initializing a group model version number of the client group when each communication round starts, and sending group model parameters, the group model version number, the preset model outdated threshold and the preset total time budget per round to all the clients of the group in each communication round;
step S22, initializing the version number of the local model of the client when the current communication wheel starts, receiving the group model version number sent by the scheduling process of the group, the preset model outdated threshold value and the preset total time budget of each wheel, and updating the version number of the local model according to the group model version number when the group model version number sent by the scheduling process is detected at any moment of the current communication wheel;
step S23, selecting a corresponding updating mode to update the parameters and the version number of the local model according to the difference between the group model parameters, the preset model outdated threshold value, the version number of the local model and the group model version number in the time of the total time budget of each preset round of the current communication round, sending the updated parameters and the updated version number of the local model to the scheduling process of the belonging group, so that the scheduling process of the belonging group updates the group model and the group model of the belonging group, and sends the updated version number of the group model and the updated group model version number to the client of the belonging group; the updating mode comprises synchronous updating and asynchronous updating.
EXAMPLE III
The federal learning method based on semi-asynchronism provided by the third embodiment of the invention is applied to a central server, and comprises the following steps S31 to S33:
step S31, broadcasting the global model parameters to all the clients so that each client receives the global model parameters, training according to the global model parameters and local data to obtain the updating direction of the local model of the client and the updating time of the local model, and sending the updating direction of the local model and the updating time of the local model to the central server;
step S32, according to the received updating direction of the local model of each client and the received updating time of the local model, grouping each client to obtain a grouped client group, and creating a plurality of processes taking the client group as a unit to obtain a plurality of scheduling processes; each scheduling process is used for deploying a group model of a client group, initializing a group model version number of the client group in each communication round, and sending group model parameters, the group model version number, the preset model obsolescence threshold value and the preset total time budget per round to all the clients of the group in each communication round so as to enable the clients to execute local training operation;
step S33, in the current communication wheel, receiving the local training result returned by the client, so that each scheduling process updates the group model and the group model version number of the group to which the scheduling process belongs according to the local training result, and sends the updated group model and the updated group model version number to the client of the group to which the scheduling process belongs;
wherein the result of the local training is obtained by:
the client initializes the version number of a local model of the client when a current communication wheel starts, receives the group model version number, the preset model outdated threshold and the preset total time budget of each wheel sent by a scheduling process of a group to which the client belongs, and updates the version number of the local model according to the group model version number when the group model version number sent by the scheduling process is detected at any moment of the current communication wheel;
and the client selects a corresponding updating mode to update the parameters and the version number of the local model according to the difference between the group model parameters, the preset model outdated threshold, the version number of the local model and the group model version number within the time of the total time budget of each preset round of the current communication round, so as to obtain the updated parameters and version number of the local model, wherein the updating mode comprises synchronous updating and asynchronous updating.
Example four
The fourth embodiment of the invention provides a federated learning system based on semi-asynchronization, which comprises a central server, a plurality of edge computing devices and a plurality of clients, wherein the central server, the clients and the edge computing devices are in communication connection;
the central server is used for broadcasting the global model parameters to all the clients;
the client is used for receiving the global model parameters, training according to the global model parameters and local data to obtain the updating direction of the local model of the client and the updating time of the local model, and sending the updating direction of the local model and the updating time of the local model to the central server;
the central server is further configured to group each client according to the received update direction of the local model of each client and the received update time of the local model, obtain a grouped client group, send a grouping result to each client, and send a grouping deployment request to each edge computing device; wherein each edge computing device manages a client group;
the edge computing device is used for responding to a grouping deployment request of the central server, deploying a group model of a client group, initializing a group model version number of the client group at the beginning of each communication round, and sending a group model parameter, the group model version number, the preset model outdating threshold and the preset total time budget per round to all the clients of the group to which the client group belongs in each communication round;
the client is further configured to initialize a version number of a local model of the client at the beginning of each communication round, receive the group model version number, the preset model obsolescence threshold value, and the preset total time budget for each round sent by the scheduling process of the group to which the client belongs, and update the version number of the local model according to the group model version number when the group model version number sent by the scheduling process is detected at any time of each communication round; and the edge computing device is used for selecting a corresponding updating mode according to the group model parameter, the preset model outdated threshold value, the difference between the version number of the local model and the group model version number of the group to which the local model belongs and sending the updated parameter and version number of the local model to the edge computing device of the group to which the local model belongs within the preset total time budget of each communication wheel; the updating mode comprises synchronous updating and asynchronous updating.
The edge computing device is further configured to update the group model and the group model version number of the belonging group in each communication round according to the received updated local model parameter and version number of each client of the belonging group, and send the updated group model and the updated group model version number to the client of the belonging group.
Referring to fig. 5, fig. 5 is a block diagram of a semi-asynchronous federated learning system according to the fourth embodiment of the present invention, which executes steps S11 to S15 shown in fig. 1, in particular, which is directed to the training process of the long-short term memory artificial neural network model for the sentiment analysis task of the tweet user, and can alleviate the problem of the fall-behind effect and improve the model accuracy. The framework designed in the embodiment is totally divided into three layers, namely a client layer, a scheduling process layer and a server layer.
The client consists of a variety of computing devices 504 with communication and storage capabilities, including a portable personal computer, a smart phone, a sensor, a smart camera, and the like.
Each client self-maintains a tweet data set 512, which is stored in the memory of the device, including flash memory, solid state disk, hard disk drive, etc. The data in the data set 512 may be downloaded via a network or collected by a key-in device of the device, such as a tweet downloaded from social software by a smartphone and a tweet key-in, and the data will be used as a training set to participate in the neural network model training.
The client is provided with wireless communication capability 506, and such communication may be established over a cellular network, a wireless local area network (Wi-Fi). The client can send and receive data to the computing device 504 and cloud server 508 of the scheduling process layer.
The computing device 504 of the scheduling process layer is a digital computer, such as a personal computer, workstation, server, with computing, storage, and communication capabilities. The computing devices of the dispatch process layer may send and receive data with client device 502 and cloud server 508 over the internet, and may read and write to memory 514. The computing devices of the scheduling process layer run a computer operating system, which, using application programming interfaces provided by the operating system, is able to create and run a scheduling process 522, a model collection process 524, and a model integration process 526. The scheduling process 522 schedules model training tasks for client devices, which tasks include sending neural network models to the client devices 502 to be trained over the network 506, the model collection process 524 receives the neural network models transmitted by the network 506, and then saves the models to the memory 514 using the model integration process 526. A long-short term memory artificial neural network model for emotion analysis tasks comprises a two-layer LSTM binary classifier with 256 hidden units and pre-trained GloVe embedding with 300 dimensions, a 25-character sequence is used as input of the model, each character is embedded into a 300-dimensional space by searching GloVe, and after passing through a 2-layer LSTM layer and a dense connection layer, each training sample outputs one character. The neural network can achieve 69% test accuracy by training 200 rounds based on the Sentiment analysis data set sentment 140.
Cloud server 508 is a computer, such as a workstation, server, blade server. The cloud server may communicate with the computing device 504 and the client device 502 of the scheduling process layer via a digital internet with computing, storage, and communication capabilities, and may read and write to the memory 516. Cloud server 508 runs a computer operating system and, using an application programming interface provided by the operating system, creates a model collection process 524 and a model integration process 526.
The model integration process 526 at the scheduling process layer takes the neural network model at the scheduling process layer as input, then executes a model integration algorithm such as FederatedAveraging, outputs a group model, and stores the group model in the memory 414. The FederatedAveraging algorithm is as follows:
Figure BDA0003122518700000171
wherein
Figure BDA0003122518700000172
Representing the model parameters of t-round client ends K, K representing the number of client ends participating in training, n representing the size of the total training set, nkRepresenting the training set size on client k. w is atRepresenting model parameters after t-round integration
The specific execution flow is as follows:
in step S1, the cloud server 508 collects the tweet data set 512 of the client device 502 via the internet, calculates the direction of model update of each client, and counts the update time of the model of each client.
In step S2, the cloud server 508 groups the clients by using a spectral clustering algorithm based on the statistical information obtained in step S1.
Step S3, the cloud server 508 sends grouping information to the computing device 504 of the scheduling process layer through the internet, and the computing device 504 of the scheduling process layer creates a scheduling process, a model collection process, and a model integration process;
step S4, the computing device 504 of the scheduling process layer prepares for initialization work, initializes the group model version number, sets a preset model obsolescence threshold, and presets a total time budget for each round;
in step S5, the scheduling process in the computing device 504 of the scheduling process layer broadcasts the group model, the group model version number, and the preset model obsolescence threshold value to preset the total time budget per round to the client 502 via the network 506. The client 502 calculates whether the local model is outdated, if not, the client 502 updates the local model version number after training is completed, the local model and the version number thereof are saved on the memory 512, and then the local model is updated to the group model and the group model version number is updated through the model collection process and the model integration process of the scheduling process layer computing device 504, and the group model version number are saved on the memory 514. If the local model on the client 502 is out of date, i.e., synchronization information is sent to the computing device 504 of the scheduling process layer over the network 506, the computing device 504 of the scheduling process layer receives the post-scheduling model collection process and collects all the model parameters of the client 502 that still have a time budget. After all clients 502 needing training are trained, the version numbers of the local models are updated and stored in the memory 512, the computing device 504 at the scheduling process level schedules the models to integrate the process update group models and update the group model version numbers, and the group models and the group model version numbers are stored in the memory 514.
EXAMPLE five
An embodiment five of the present invention provides a central server, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, the processor implements the semi-asynchronous based federated learning method as described in embodiment three.
The Processor may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. The general purpose processor may be a microprocessor or the processor may be any conventional processor or the like, the processor is a control center of the mobile communication network traffic prediction device, and various interfaces and lines are used to connect various parts of the whole mobile communication network traffic prediction device.
The memory may be used for storing the computer programs and/or modules, and the processor may implement various functions of the mobile communication network traffic prediction apparatus by executing or executing the computer programs and/or modules stored in the memory and calling data stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone, and the like. In addition, the memory may include high speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), at least one magnetic disk storage device, a Flash memory device, or other volatile solid state storage device.
EXAMPLE six
An embodiment of the present invention provides a computer-readable storage medium, where the computer-readable storage medium includes a stored computer program, and when the computer program runs, a device where the computer-readable storage medium is located is controlled to execute the federal learning method based on semi-asynchronization in embodiment two or embodiment three.
Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.
While the foregoing is directed to the preferred embodiment of the present invention, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention.

Claims (8)

1. A federated learning method based on semi-asynchronization is characterized by comprising the following steps:
the central server broadcasts the global model parameters to all the clients;
each client receives the global model parameters, trains according to the global model parameters and local data, obtains the updating direction of the local model of the client and the updating time of the local model, and sends the updating direction of the local model and the updating time of the local model to the central server;
the central server groups each client according to the received updating direction of the local model of each client and the received updating time of the local model to obtain a grouped client group, and creates a plurality of processes taking the client group as a unit to obtain a plurality of scheduling processes; or, the central server groups each client according to the received update direction of the local model of each client and the update time of the local model, so as to obtain a grouped client group, and sends a grouping deployment request to each edge computing device, so that each edge computing device responds to the grouping deployment request to create a scheduling process of the group to which the edge computing device belongs; each scheduling process is used for deploying a group model of a client group, initializing a group model version number of the client group when each communication round starts, and sending group model parameters, the group model version number, a preset model outdated threshold value and a preset total time budget per round to all the clients of the group in each communication round;
for each client, initializing the version number of a local model of the client when the current communication wheel starts, receiving the group model version number sent by a scheduling process of a group to which the client belongs, the preset model outdated threshold value and the preset total time budget per wheel, and updating the version number of the local model according to the group model version number when the group model version number sent by the scheduling process is detected at any moment of the current communication wheel;
each client selects a corresponding updating mode to update the parameters and the version number of the local model according to the difference between the group model parameters, the preset model outdated threshold value, the version number of the local model and the group model version number in the time of the preset total time budget of each communication wheel, and sends the updated parameters and the updated version number of the local model to the scheduling process of the group, so that the scheduling process of the group updates the group model and the group model version number of the group, and sends the updated group model and the updated group model version number to the clients of the group; the updating mode comprises synchronous updating and asynchronous updating.
2. The semi-asynchronous federated learning method according to claim 1, wherein each of the clients selects a corresponding update manner to update the parameters and the version number of the local model according to the group model parameters, the preset model obsolete threshold, and the difference between the version number of the local model and the group model version number within the time of the preset total time budget for each communication round of the current communication round, and sends the updated parameters and the version number of the local model to the scheduling process of the belonging group, so that the scheduling process of the belonging group updates the group model and the group model version number of the belonging group, and sends the updated group model and the updated group model version number to the clients of the belonging group, which specifically includes:
for each client, when the client detects that the difference between the version number of the local model and the group model version number of the affiliated group is smaller than the preset model outdated threshold value within the preset total time budget of each communication wheel, the client performs local training according to local data and received group model parameters of the affiliated group to obtain updated parameters and version numbers of the local model, and asynchronously sends the updated parameters and version numbers of the local model to the scheduling process of the affiliated group, so that the scheduling process of the affiliated group updates the group model and the group model version number of the affiliated group, and sends the updated group model and the updated group model version number to all clients requesting group model information of the affiliated group;
when the client detects that the difference between the version number of the local model and the version number of the group model of the belonging group is larger than the preset model passing threshold value in the current communication wheel, sending a synchronous updating request to the scheduling process of the belonging group, so that the scheduling process of the belonging group responds to the synchronous updating request and sends a synchronous updating command to all the clients of the belonging group with time budget;
and all the clients receiving the synchronous updating command execute local training operation, synchronously update the parameters and the version numbers of the local models, send the updated version numbers of the local models and the updated local model parameters to the scheduling process of the affiliated group, so that the scheduling process of the affiliated group updates the group models and the group model version numbers of the affiliated group, and send the updated group models and the updated group model version numbers to all the clients of the affiliated group with the time budget.
3. The semi-asynchronous federated learning method of claim 1, wherein the central server groups each of the clients according to the received update direction of the local model of each of the clients and the update time of the local model to obtain a grouped client group, specifically comprising:
and the central server groups each client by adopting a clustering algorithm according to the received updating direction of the local model of each client and the received updating time of the local model, so as to obtain a grouped client group.
4. A federated learning method based on semi-asynchronization is applied to a client and is characterized by comprising the following steps:
receiving the global model parameters broadcasted by the central server, training according to the global model parameters and local data to obtain the updating direction of the local model of the client and the updating time of the local model, and sending the updating direction of the local model and the updating time of the local model to the central server so that the central server executes the operation of grouping the clients and creating a scheduling process; wherein the client grouping operation is to: the central server groups each client according to the received updating direction of the local model of each client and the received updating time of the local model, so as to obtain a grouped client group; the operation of the central server for establishing the scheduling process is as follows: creating a plurality of processes taking a client group as a unit according to the grouping result of the clients to obtain a plurality of scheduling processes; each scheduling process is used for deploying a group model of a client group, initializing a group model version number of the client group when each communication round starts, and sending group model parameters, the group model version number, a preset model outdated threshold value and a preset total time budget per round to all the clients of the group in each communication round;
initializing the version number of a local model when a current communication wheel starts, receiving the version number of the group model, the outdated threshold value of the preset model and the total time budget of each preset wheel, which are sent by the scheduling process of the belonging group, and updating the version number of the local model according to the version number of the group model when the version number of the group model, which is sent by the scheduling process, is detected at any moment of the current communication wheel;
selecting a corresponding updating mode to update the parameters and the version number of the local model according to the group model parameters, the preset model outdated threshold value, the difference between the version number of the local model and the group model version number in the time of the preset total time budget of each communication wheel, and sending the updated parameters and the updated version number of the local model to the scheduling process of the belonging group, so that the scheduling process of the belonging group updates the group model and the group model version number of the belonging group, and sends the updated group model and the updated group model version number to the client of the belonging group; the updating mode comprises synchronous updating and asynchronous updating.
5. A federated learning method based on semi-asynchronization is applied to a central server, and is characterized by comprising the following steps:
broadcasting the global model parameters to all the clients so that each client receives the global model parameters, training according to the global model parameters and local data to obtain the updating direction of the local model of the client and the updating time of the local model, and sending the updating direction of the local model and the updating time of the local model to the central server;
grouping each client according to the received updating direction of the local model of each client and the received updating time of the local model to obtain a grouped client group, and creating a plurality of processes taking the client group as a unit to obtain a plurality of scheduling processes; each scheduling process is used for deploying a group model of a client group, initializing a group model version number of the client group when each communication round starts, and sending group model parameters, the group model version number, a preset model outdated threshold value and a preset total time budget per round to all the clients of the group in each communication round so as to enable the clients to execute local training operation;
in the current communication wheel, receiving a local training result returned by the client, so that each scheduling process updates the group model and the group model version number of the group to which the scheduling process belongs according to the local training result, and sends the updated group model and the updated group model version number to the client of the group to which the scheduling process belongs;
wherein the result of the local training is obtained by:
the client initializes the version number of a local model of the client when a current communication wheel starts, receives the group model version number, the preset model outdated threshold and the preset total time budget of each wheel sent by a scheduling process of a group to which the client belongs, and updates the version number of the local model according to the group model version number when the group model version number sent by the scheduling process is detected at any moment of the current communication wheel;
and the client selects a corresponding updating mode to update the parameters and the version number of the local model according to the difference between the group model parameters, the preset model outdated threshold, the version number of the local model and the group model version number within the time of the total time budget of each preset round of the current communication round, so as to obtain the updated parameters and version number of the local model, wherein the updating mode comprises synchronous updating and asynchronous updating.
6. A central server comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, the processor implementing the semi-asynchronous based federated learning method of claim 5 when executing the computer program.
7. A semi-asynchronous based federated learning system comprising a central server, a plurality of edge computing devices, and a plurality of clients, the central server, the clients, and the edge computing devices being communicatively connected;
the central server is used for broadcasting the global model parameters to all the clients;
the client is used for receiving the global model parameters, training according to the global model parameters and local data to obtain the updating direction of the local model of the client and the updating time of the local model, and sending the updating direction of the local model and the updating time of the local model to the central server;
the central server is further configured to group each client according to the received update direction of the local model of each client and the received update time of the local model, obtain a grouped client group, send a grouping result to each client, and send a grouping deployment request to each edge computing device; wherein each edge computing device manages a client group;
the edge computing device is used for responding to a grouping deployment request of the central server, deploying a group model of a client group, initializing a group model version number of the client group at the beginning of each communication round, and sending group model parameters, the group model version number, a preset model outdating threshold value and a preset total time budget per round to all the clients of the group to which the client belongs in each communication round;
the client is further configured to initialize a version number of a local model of the client at the beginning of each communication round, receive the group model version number, the preset model obsolescence threshold value, and the preset total time budget per round sent by the edge computing device of the group to which the client belongs, and update the version number of the local model according to the group model version number when the group model version number sent by the scheduling process is detected at any time of each communication round; and the edge computing device is used for selecting a corresponding updating mode according to the group model parameter, the preset model outdated threshold value, the difference between the version number of the local model and the group model version number of the group to which the local model belongs and sending the updated parameter and version number of the local model to the edge computing device of the group to which the local model belongs within the preset total time budget of each communication wheel; the updating mode comprises synchronous updating and asynchronous updating.
The edge computing device is further configured to update the group model and the group model version number of the belonging group in each communication round according to the received updated local model parameter and version number of each client of the belonging group, and send the updated group model and the updated group model version number to the clients of the belonging group.
8. A computer-readable storage medium comprising a stored computer program, wherein the computer program, when executed, controls an apparatus on which the computer-readable storage medium is located to perform the semi-asynchronous based federated learning method of claim 4 or 5.
CN202110681063.8A 2021-06-18 2021-06-18 Federal learning method, system, server and medium based on semi-asynchronization Active CN113516249B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110681063.8A CN113516249B (en) 2021-06-18 2021-06-18 Federal learning method, system, server and medium based on semi-asynchronization

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110681063.8A CN113516249B (en) 2021-06-18 2021-06-18 Federal learning method, system, server and medium based on semi-asynchronization

Publications (2)

Publication Number Publication Date
CN113516249A true CN113516249A (en) 2021-10-19
CN113516249B CN113516249B (en) 2023-04-07

Family

ID=78065644

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110681063.8A Active CN113516249B (en) 2021-06-18 2021-06-18 Federal learning method, system, server and medium based on semi-asynchronization

Country Status (1)

Country Link
CN (1) CN113516249B (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115329990A (en) * 2022-10-13 2022-11-11 合肥本源物联网科技有限公司 Asynchronous federated learning acceleration method based on model segmentation under edge calculation scene
CN115511103A (en) * 2022-10-20 2022-12-23 抖音视界有限公司 Method, apparatus, device and medium for federal learning
CN115618963A (en) * 2022-10-20 2023-01-17 重庆移通学院 Wireless federal learning asynchronous training method based on optimized direction guidance
CN117150416A (en) * 2023-10-27 2023-12-01 烟台大学 Method, system, medium and equipment for detecting abnormal nodes of industrial Internet

Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200050951A1 (en) * 2018-08-09 2020-02-13 International Business Machines Corporation Collaborative distributed machine learning
US20200293887A1 (en) * 2019-03-11 2020-09-17 doc.ai, Inc. System and Method with Federated Learning Model for Medical Research Applications
CN111708640A (en) * 2020-06-23 2020-09-25 苏州联电能源发展有限公司 Edge calculation-oriented federal learning method and system
CN112257876A (en) * 2020-11-15 2021-01-22 腾讯科技（深圳）有限公司 Federal learning method, apparatus, computer device and medium
CN112348199A (en) * 2020-10-30 2021-02-09 河海大学 Model training method based on federal learning and multi-task learning
CN112508205A (en) * 2020-12-04 2021-03-16 中国科学院深圳先进技术研究院 Method, device and system for scheduling federated learning
CN112532451A (en) * 2020-11-30 2021-03-19 安徽工业大学 Layered federal learning method and device based on asynchronous communication, terminal equipment and storage medium
US20210097430A1 (en) * 2019-09-30 2021-04-01 International Business Machines Corporation Data protection distributed learning
CN112633146A (en) * 2020-12-21 2021-04-09 杭州趣链科技有限公司 Multi-pose face gender detection training optimization method and device and related equipment
CN112668726A (en) * 2020-12-25 2021-04-16 中山大学 Personalized federal learning method with efficient communication and privacy protection
CN112836865A (en) * 2021-01-19 2021-05-25 南京邮电大学 Airport airplane flow asynchronous prediction method based on federal learning
CN112966832A (en) * 2021-03-31 2021-06-15 上海嗨普智能信息科技股份有限公司 Multi-server-based federal learning system
US20210182740A1 (en) * 2019-12-17 2021-06-17 Lg Electronics Inc. Artificial intelligence server and method for updating artificial intelligence model by merging plurality of pieces of update information
WO2021119601A1 (en) * 2019-12-13 2021-06-17 Qualcomm Technologies, Inc. Federated mixture models

Patent Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200050951A1 (en) * 2018-08-09 2020-02-13 International Business Machines Corporation Collaborative distributed machine learning
US20200293887A1 (en) * 2019-03-11 2020-09-17 doc.ai, Inc. System and Method with Federated Learning Model for Medical Research Applications
US20210097430A1 (en) * 2019-09-30 2021-04-01 International Business Machines Corporation Data protection distributed learning
WO2021119601A1 (en) * 2019-12-13 2021-06-17 Qualcomm Technologies, Inc. Federated mixture models
US20210182740A1 (en) * 2019-12-17 2021-06-17 Lg Electronics Inc. Artificial intelligence server and method for updating artificial intelligence model by merging plurality of pieces of update information
CN111708640A (en) * 2020-06-23 2020-09-25 苏州联电能源发展有限公司 Edge calculation-oriented federal learning method and system
CN112348199A (en) * 2020-10-30 2021-02-09 河海大学 Model training method based on federal learning and multi-task learning
CN112257876A (en) * 2020-11-15 2021-01-22 腾讯科技（深圳）有限公司 Federal learning method, apparatus, computer device and medium
CN112532451A (en) * 2020-11-30 2021-03-19 安徽工业大学 Layered federal learning method and device based on asynchronous communication, terminal equipment and storage medium
CN112508205A (en) * 2020-12-04 2021-03-16 中国科学院深圳先进技术研究院 Method, device and system for scheduling federated learning
CN112633146A (en) * 2020-12-21 2021-04-09 杭州趣链科技有限公司 Multi-pose face gender detection training optimization method and device and related equipment
CN112668726A (en) * 2020-12-25 2021-04-16 中山大学 Personalized federal learning method with efficient communication and privacy protection
CN112836865A (en) * 2021-01-19 2021-05-25 南京邮电大学 Airport airplane flow asynchronous prediction method based on federal learning
CN112966832A (en) * 2021-03-31 2021-06-15 上海嗨普智能信息科技股份有限公司 Multi-server-based federal learning system

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
NISHIO T等: "Client selection for federated learning with heterogeneous resources in mobile edge", 《PROCEEDINGS OF THE 2019 ICC 2019-2019IEEE INTERNATIONAL CONFERENCE ON COMMUNICATIONS》 *
邱鑫源等: "联邦学习通信开销研究综述", 《HTTPS://KNS.CNKI.NET/KCMS/DETAIL/51.1307.TP.20210430.1311.004.HTML》 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115329990A (en) * 2022-10-13 2022-11-11 合肥本源物联网科技有限公司 Asynchronous federated learning acceleration method based on model segmentation under edge calculation scene
CN115329990B (en) * 2022-10-13 2023-01-20 合肥本源物联网科技有限公司 Asynchronous federated learning acceleration method based on model segmentation under edge computing scene
CN115511103A (en) * 2022-10-20 2022-12-23 抖音视界有限公司 Method, apparatus, device and medium for federal learning
CN115618963A (en) * 2022-10-20 2023-01-17 重庆移通学院 Wireless federal learning asynchronous training method based on optimized direction guidance
CN117150416A (en) * 2023-10-27 2023-12-01 烟台大学 Method, system, medium and equipment for detecting abnormal nodes of industrial Internet
CN117150416B (en) * 2023-10-27 2024-03-08 烟台大学 Method, system, medium and equipment for detecting abnormal nodes of industrial Internet

Also Published As

Publication number Publication date
CN113516249B (en) 2023-04-07

Similar Documents

Publication Publication Date Title
CN113516249B (en) Federal learning method, system, server and medium based on semi-asynchronization
US20240119286A1 (en) Adaptive artificial neural network selection techniques
CN108108821B (en) Model training method and device
CN109993300B (en) Training method and device of neural network model
CN111602148A (en) Regularized neural network architecture search
CN113177538B (en) Video cycle identification method and device, computer equipment and storage medium
CN109961032B (en) Method and apparatus for generating classification model
US11494614B2 (en) Subsampling training data during artificial neural network training
CN111105006A (en) Deep learning network training system and method
CN111492394A (en) Attendee engagement determination systems and methods
CN111104954A (en) Object classification method and device
CN115130711A (en) Data processing method and device, computer and readable storage medium
CN111243604B (en) Training method for speaker recognition neural network model supporting multiple awakening words, speaker recognition method and system
CN116126354A (en) Model deployment method, device, electronic equipment and storage medium
CN111222399A (en) Method and device for identifying object identification information in image and storage medium
CN115718868A (en) Model training method, device and system
CN113827978A (en) Loss user prediction method and device and computer readable storage medium
CN114332550A (en) Model training method, system, storage medium and terminal equipment
CN110415318B (en) Image processing method and device
CN113362852A (en) User attribute identification method and device
CN115620074A (en) Image data classification method, device and medium
CN113312445B (en) Data processing method, model construction method, classification method and computing equipment
CN115238806A (en) Sample class imbalance federal learning method and related equipment
CN112820302A (en) Voiceprint recognition method and device, electronic equipment and readable storage medium
CN113568943A (en) Book resource processing method, device, equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant