CN115130683A - An asynchronous federated learning method and system based on a multi-agent model - Google Patents

Abstract

The invention relates to the technical field of asynchronous federated learning, and provides an asynchronous federated learning method and system based on a multi-agent model, including: randomly selecting several pre-training clients in each group of clients, and obtaining each pre-training client The decision result of whether to participate in the training and uploading of the model; receive the local model obtained from the training of each group of clients participating in the training of the model and the uploaded client to obtain the group model; perform weighted aggregation on the group model to obtain the global model. It can not only solve the long waiting delay problem in synchronous federated learning, but also solve the communication bottleneck problem in fully semi-asynchronous federated learning.

Description

An asynchronous federated learning method and system based on a multi-agent model

technical field

The invention belongs to the technical field of asynchronous federated learning, and in particular relates to an asynchronous federated learning method and system based on a multi-agent model.

Background technique

The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

Federated learning algorithms involve hundreds of millions of remote devices training locally on data generated by their devices and collectively training a global, shared model under the coordination of a central server acting as an aggregator.

The proposal of federated average algorithm is the beginning of federated learning, which breaks the inherent mode of traditional centralized and distributed training, and protects the privacy of data by only passing model gradient parameters. Experiments show that this method can realize flexible and efficient communication and reduce communication costs. However, due to the highly ideal environment requirements when the federated averaging algorithm runs, there are also many problems in real heterogeneous client scenarios. The most typical problem is that in each round of training, the client that trains faster needs to wait for the slowest client. The client can upload the aggregated update global model only after completing the training. The training efficiency of the entire federated learning is determined by the slowest client, which will greatly reduce the training efficiency of the model and prolong the training time.

For synchronous federated learning like federated averaging, the experiment is carried out in a highly ideal scenario, while in real scenarios, due to the heterogeneity of devices and the unreliability of the network, there will inevitably be some laggards (lag) equipment or equipment that exits training), so in practical scenarios, an asynchronous federated training method is used, in which the server does not need to wait for the lagging equipment to aggregate.

There is currently an exponentially weighted average asynchronous federated learning algorithm. Different clients use their own local data sets to train their own local models. Once a client completes the training, it immediately sends the model parameters to the central server, and then the central server immediately Aggregate model parameters without waiting for any other edge device. Its core idea is that the later the local model is transmitted, the lower the weight is given to it, which can adaptively balance the convergence speed and variance reduction. However, it still cannot solve the inherent problem in fully asynchronous federated learning, that is, the communication bottleneck caused by the frequent communication between the local client and the central server.

For the training waiting delay problem in synchronous federated learning and the communication bottleneck problem in fully asynchronous federated learning, there is also a method to propose a semi-asynchronous method to neutralize these two huge problems, but in real life, it is more difficult to In complex heterogeneous client scenarios, there is no good method to minimize communication overhead and training delay without losing model accuracy.

SUMMARY OF THE INVENTION

In order to solve the technical problems existing in the above background technology, the present invention provides an asynchronous federated learning method and system based on a multi-agent model, which can not only solve the long waiting delay problem in synchronous federated learning, but also solve the problem of complete semi-asynchronous federation Communication bottlenecks in learning.

In order to achieve the above object, the present invention adopts the following technical solutions:

A first aspect of the present invention provides an asynchronous federated learning method based on a multi-agent model, which includes:

Randomly select several pre-training clients in each group of clients, and obtain the decision result of whether each pre-training client participates in model training and uploading;

Receive the local model obtained from the training of the model and the uploaded client in each group of clients, and obtain the group model;

A weighted aggregation of the group models results in a global model.

Further, the acquisition method of the decision result is:

Get the status of each pretrained client;

The state of each pre-training client is input to the reinforcement learning agent network, and the decision result of whether each pre-training client participates in the training and uploading of the model is obtained.

Further, the state of the pre-training client includes: the training round index t, the amount of data on the client, the number of times the client has participated in local model update and uploading by the t round, and the location of the client by the t round. The number of updates to the group model, the communication overhead for all pretrained clients, and the training latency for all pretrained clients.

Further, the reinforcement learning agent network aims to maximize cumulative return.

Further, when the group models are weighted and aggregated, the weight of each group model is related to the number of updates of the group model of each group.

A second aspect of the present invention provides an asynchronous federated learning system based on a multi-agent model, comprising:

a client intelligent selection module, which is configured to: randomly select several pre-training clients in each group of clients, and obtain the decision result of whether each pre-training client participates in the training and uploading of the model;

An intra-group synchronous training module, which is configured to: receive a local model obtained from the training of the participating models in each group of clients and the uploaded client-side training to obtain a group model;

An asynchronous training module between groups, which is configured to: perform weighted aggregation on the group model to obtain a global model.

Further, the client intelligent selection module is specifically configured as:

Get the status of each pretrained client;

The state of each pre-training client is input to the reinforcement learning agent network, and the decision result of whether each pre-training client participates in the training and uploading of the model is obtained.

Further, when the group models are weighted and aggregated, the weight of each group model is related to the number of updates of the group model of each group.

A third aspect of the present invention provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the steps in the above-mentioned asynchronous federated learning method based on a multi-agent model .

A fourth aspect of the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor implementing the above-mentioned one when executing the program Steps in an asynchronous federated learning approach based on a multi-agent model.

Compared with the prior art, the beneficial effects of the present invention are:

The invention provides an asynchronous federated learning method based on a multi-agent model, which introduces a multi-agent reinforcement learning to execute an efficient client intelligent selection mode, which is used to replace the random selection strategy in the previous federated learning method, and not only optimizes the model accuracy , and greatly improves the efficiency of model training. Compared with other more advanced semi-asynchronous federated learning methods, when the training reaches the specified accuracy, it costs less communication overhead and training delay, and can be applied to a wide range of heterogeneous clients in real life. Machine learning model training scenarios.

The present invention provides an asynchronous federated learning method based on a multi-agent model. Each client has a reinforcement learning agent, which will decide whether to participate in the training and uploading and aggregation of the current round of models according to its own observations, which can solve complex problems. The machine learning model training problem in the heterogeneous client scenario can not only solve the long waiting delay problem in synchronous federated learning, but also solve the communication bottleneck problem in fully semi-asynchronous federated learning.

Description of drawings

The accompanying drawings forming a part of the present invention are used to provide further understanding of the present invention, and the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, and do not constitute an improper limitation of the present invention.

1 is an overall flow chart of an asynchronous federated learning method based on a multi-agent model according to Embodiment 1 of the present invention;

Fig. 2 is the accuracy change diagram under the complex scene on the F-MNIST data set of Embodiment 1 of the present invention;

Fig. 3 (a) is the communication overhead diagram under the complex scene on the F-MNIST data set of Embodiment 1 of the present invention;

Fig. 3 (b) is the training delay diagram under the complex scene on the F-MNIST data set according to the first embodiment of the present invention;

FIG. 4 is a graph of the accuracy change in a complex scene on the CIFAR-10 dataset according to Embodiment 1 of the present invention;

Figure 5(a) is a communication overhead diagram in a complex scenario on the CIFAR-10 data set according to Embodiment 1 of the present invention;

Fig. 5(b) is a training delay diagram in a complex scene on the CIFAR-10 dataset according to Embodiment 1 of the present invention.

Detailed ways

The present invention will be further described below with reference to the accompanying drawings and embodiments.

It should be noted that the following detailed description is exemplary and intended to provide further explanation of the invention. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

It should be noted that the terminology used herein is for the purpose of describing specific embodiments only, and is not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly dictates otherwise, the singular is intended to include the plural as well, furthermore, it is to be understood that when the terms "comprising" and/or "including" are used in this specification, it indicates that There are features, steps, operations, devices, components and/or combinations thereof.

Terminology Explanation:

The objective function of the federated learning algorithm:

是客户端k的本地经验损失，l_i(x_i,y_i；w)是数据样本{x_i,y_i}对应的损失函数值，w为需要训练的机器学习模型；K是客户端总数，D_k(k∈{1,...,K})表示存储在本地客户端k上的数据样本，n_k＝|D_k|表示客户端k上的数据样本数量；

为存储在K个客户端上的总数据样本数量；假设对于对于任何k≠k'，

in,

is the local experience loss of client k, _{li (x i , y i ; w} ) is the loss function value corresponding to the data sample {x _i , y _i }, w is the machine learning model to be trained; K is the total number of clients , D _k (k∈{1,...,K}) represents the data samples stored on the local client k, n _k = |D _k | represents the number of data samples on the client k;

is the total number of data samples stored on K clients; suppose that for any k≠k',

The ultimate goal of the federated learning algorithm: find a model w _* that minimizes the objective function:

w _* = arg min f(w)

Federated Average Algorithm: A common method for solving the optimization problem defined in the ultimate goal of a federated learning algorithm with simultaneous updates in a nonconvex setting. This approach trains the model by randomly sampling a subset of clients with a certain probability at each round, each local client using an optimizer (such as stochastic gradient descent) for several local iterations with its own data.

The update method of the global model in the exponentially weighted average asynchronous federated learning algorithm:

α _t ←α×s(t-τ)

w ^t ←(1-α _t )w ^t-1 +α _t w ^new

Among them, τ is the round index when the fastest client uploads and updates the global model, t is the current round index, α∈(0,1) is the mixing coefficient, α _t is the dynamically updated coefficient of the current round t, w ^t-1 is the old model obtained from the previous round of training, w ^new is the new model obtained by the current training, w ^t is the new model obtained by the weighted update for the next training, s( ) is the old model degree function, which can take values

Example 1

This embodiment provides an asynchronous federated learning method based on a multi-agent model, as shown in Figure 1, which specifically includes the following steps:

Step 1. Client intelligent selection stage. During each round (t-th round) of training, randomly select several (|P|) pre-training clients in each group of clients group _m , obtain the status of each pre-training client, and compare the status of each pre-training client The state is input to the reinforcement learning agent network, and the decision result of whether each pre-training client participates in the training and uploading of the model is obtained.

组模型的更新需要等待组内最慢的客户端完成本地训练。Suppose there is now a federated learning task that divides all clients into M groups, and a formal description is made for one of the groups, group _m , and the other groups are similar. group _m randomly selects |P| pre-training clients in each training round t, and then |P| clients decide whether to participate in the training and uploading of the model according to their own reinforcement learning agents. After the decision, there will be | P'| clients perform local synchronous federated training and model upload updates. The communication overhead of each communication (uploading or downloading model) between client n and the central server is a fixed value B _n , and the response time CP _n of client n is represented by the number of global rounds experienced by the client for one local training , the larger the value of CP _n , the longer the local training time of the client, the slower the response, and the training delay of client n.

The update of the group model needs to wait for the slowest client in the group to finish local training.

Step 101: The clients selected for pre-training obtain their respective current states:

到第t轮为止客户端n所在的group_m组模型更新次数为

所有预训练客户端的通信开销B^t＝{B_j|j∈P}、所有预训练客户端的训练延迟

The state space of each reinforcement learning agent n consists of six parts: the index t of the current training round, the size of the data on the corresponding client n |D _n |, and the client n participating in the local Number of model updates and uploads

Up to the t round, the number of model updates of group _m where client n belongs is:

Communication overhead of all pre-training clients B ^t = {B _j |j∈P}, training delay of all pre-training clients

Step 102: Input the current state into the reinforcement learning agent network to obtain the decision result:

Among them, 1 indicates that the client participates in the training and uploading of the model in this round, and 0 indicates that it does not participate.

Among them, the reinforcement learning agent network aims to maximize the cumulative return, specifically:

After completing the training in the group, upload the group model to update the global model and then calculate the reward:

表示第t轮智能选择的所有客户端的通信开销之和，

表示第t轮智能选择的所有客户端的训练延迟之和；Among them, u is a constant greater than 1, and an appropriate value will be selected according to the experimental conditions, acc _t represents the accuracy of the global model updated after the t-th round of selecting a suitable client for training, and acc _last represents the latest global model accuracy,

represents the sum of the communication overheads of all clients intelligently selected in the t-th round,

represents the sum of the training delays of all clients intelligently selected in round t;

The reinforcement learning agent network will be trained to maximize the expected cumulative return R, which is described as follows:

where E is the total number of updates of the global model, and γ∈(0,1] is the discount factor for future rewards.

Step 2: Intra-group synchronous training phase. Receive the local model obtained from the training of the model and the uploaded client in each group of clients to obtain the group model.

An intra-group decision for model training and uploading of all clients will perform synchronous federated training to get the group model:

分别表示在第t轮group_m智能选择的客户端子集、智能选择的客户端数量、客户端k上的数据量、P_t'中的所有数据量、学习率、客户端k的本地经验损失的梯度。Among them, P _t ', |P _t '|, n _k , N _c , η,

represent the subset of clients intelligently selected by group _m in round t, the number of intelligently selected clients, the amount of data on client k, the amount of all data in P _t ', the learning rate, and the local experience loss of client k, respectively gradient.

Step 3, the asynchronous training phase between groups. A weighted aggregation of the group models results in a global model. When weighted aggregation of group models is performed, the weight of each group model is related to the number of updates of the group model for each group.

Assuming that all clients are divided into M groups, so far, the update times of each group are T ₁ , T ₂ ,..., T _M respectively, and the total number of updates for all groups is T ₁ +T ₂ +...+T _M = T, then the description of the global model obtained by weighted aggregation is as follows:

为group_m对应的权重，

根据这个公式相对较慢的组M+1-m的值更小，T_(M+1-m)值更大，从而被分配一个更大的权重值。in,

is the weight corresponding to group _m ,

According to this formula, the relatively slow group M+1-m has a smaller value and a larger T _(M+1-m) value, and thus is assigned a larger weight value.

In order to solve the long waiting delay problem in the synchronous federated learning and the communication bottleneck problem in the fully semi-asynchronous federated learning, the present embodiment proposes an asynchronous federated learning method based on a multi-agent model (abbreviation is MAAFL), different from other semi-asynchronous methods, the present invention introduces multi-agent reinforcement learning to perform efficient intelligent selection of intra-group clients, each client has a reinforcement learning agent, which will decide whether to Participate in this round of model training and upload aggregation; this invention combines multi-agent reinforcement learning to execute efficient client-side intelligent selection strategies in asynchronous scenarios, which not only optimizes model accuracy, but also greatly improves model training efficiency, which is more efficient than others. Advanced semi-asynchronous federated learning methods spend less communication overhead and training latency when training to a specified accuracy.

This embodiment compares MAAFL with the synchronous federated averaging method (abbreviated as FedAvg), the fully asynchronous asynchronous federated method (abbreviated as FedAsync) and the layered semi-asynchronous method (abbreviated as FedAT).

Simultaneous federated averaging method (abbreviated as FedAvg) is a baseline federated learning method. In each round, a certain percentage of all clients are randomly selected for training, and the server averagely aggregates the weights received from the selected clients.

Fully Asynchronous Asynchronous Federated Approach (FedAsync for short) is a baseline asynchronous federated learning method that uses weighted average to update the global model of the server. Different from the previous synchronous federated learning method, all clients are trained at the same time. When the server receives weights from any client, it immediately weights these weights with the weights of the current global model to obtain the latest global model, which is then compared with all currently available weights. client communication for training

Hierarchical Semi-Asynchronous Method (FedAT for short) is a semi-asynchronous federated learning method that combines synchronous intra-layer training and cross-layer asynchronous training. In the layer, a random selection strategy is used to select some clients for synchronous federation training, and the layers communicate with the central server in an asynchronous manner to update the global model.

In order to verify the effectiveness of the asynchronous federated learning method based on the multi-agent model, the experiments compare the model accuracy trained by FedAvg, FedAsync, FedAT, and MAAFL, as well as the communication overhead and training delay to achieve the specified accuracy.

In the experiments, three datasets are used, namely MNIST, Fashion-MNIST, CIFAR-10, and MNIST is a handwriting recognition dataset containing a training set of 60,000 samples and a test set of 10,000 samples. Each example is a 28 × 28 dimensional grayscale image associated with labels from 10 classes; Fashion-MNIST is a dataset of clothing images with a training set of 60,000 samples and a test set of 10,000 samples. Each example is a 28×28 dimensional grayscale image associated with labels from 10 classes; the CIFAR-10 dataset contains 60,000 32×32 dimensional color images divided into 10 classes of 6000 There are 5000 images for training and 1000 images for testing.

Experiments evaluate the models on these three different datasets, all using non-IID data partitioning. Specifically, firstly, all data of each dataset is divided into 200 fragments according to the category labels. Among the 10 category labels, all the data corresponding to each label are divided into 20 fragments, which are allocated to each local client. 2 shards belonging to different tag categories. This division is to ensure that among the 100 local clients, each client has only data of two category labels, and the data size on each client is different, thus ensuring that the training is on non-IID data carried out in the environment.

In the experiment, the local data of each client is randomly divided into 80% training set and 20% test set. For intra-group synchronous training, the same sampling method as the federated average is used. MAAFL randomly selects a part of clients as pre-training clients. Afterwards, each pre-training client performs an intelligent decision to decide whether to participate in this round of training, using stochastic gradient descent (SGD) as a local optimizer. For all datasets, the training configuration parameters for the local model are uniform: the learning rate is 0.01, the number of local iterations is 3, the batch size is 10, and the number of randomly selected pre-training clients is set to 20 for all algorithms.

The experiment simulates different performance groups. First, all clients are equally divided into 5 groups, and then the clients in each group are randomly assigned response times of 1 to 3 rounds, 3 to 5 rounds, 5 to 7 rounds, and 7 to 9 rounds. rounds, 9 to 11 rounds. Furthermore, to simulate unstable network connections, for all tests run, 20 "unstable" clients are randomly selected that drop out of training with some small probability during training. Once a client quits, it never comes back to rejoin federation training.

In the experiment, the communication overhead of each communication between each local client and the central server (uploading or downloading the model) is fixed. In order to simulate the heterogeneity of local clients in the actual scene, each client is randomly assigned a fixed range. Fixed communication cost value, different clients have different communication cost. The total communication overhead is the sum of the communication overhead between all clients and the central server during the entire training process.

In the experiment, due to the differences in the computing power of different local clients and other factors, there is a delay in uploading each round of local training models. Among the local clients that are intelligently selected for training in one round, fast clients need to wait for slow clients. For example, a client with a response time of 2 rounds needs to wait for a client with a response time of 5 rounds and a delay of 3 rounds. Global rounds. The total training latency is the accumulation of the training latency of each client during the entire training process.

The algorithm MAAFL and the three comparison methods were evaluated on three datasets. Each algorithm was run three times on each dataset, each time running until the model converged to obtain the best global model accuracy, and the three best results were averaged to obtain the best global model accuracy. The average precision of the best global model, and the standard deviation of the three results obtained by calculating the results are shown in Table 1.

Table 1. Accuracy performance of different algorithms on three datasets

The communication overhead and training latency spent to achieve the specified accuracy on the three datasets are shown in Tables 2, 3 and 4.

Table 2. Communication overhead and training delay for different algorithms to achieve the specified accuracy on the MNIST dataset

Table 3. Communication overhead and training delay for different algorithms to achieve the specified accuracy on the F-MNIST dataset

Table 4. Communication overhead and training delay for different algorithms to achieve the specified accuracy on the CIFAR-10 dataset

The experiment further simulates a more complex scenario, setting the response time of the slowest client to be 21 rounds behind, and the experimental results obtained are shown in Figure 2, Figure 3(a), Figure 3(b), Figure 4, Figure 5(a) and shown in Figure 5(b).

From these experimental results, it can be seen that in general scenarios, although the overall performance of the MAAFL method is inferior to that of FedAT, which is also a semi-asynchronous method, it also has its own advantages, and in more complex scenarios, MAAFL surpasses both in accuracy and training efficiency. FedAT. The highlights of the experimental results are summarized as follows:

(1) Compared with FedAvg, MAAFL can well solve the training delay problem in FedAvg, which is of great significance in realistic heterogeneous scenarios and avoids the long waiting process between clients;

(2) Compared with FedAsync, MAAFL avoids the communication bottleneck problem and solves the communication bottleneck problem caused by frequent communication with the central server in the completely asynchronous method;

(3) Compared with FedAT, which is also a semi-asynchronous method, MAAFL can highlight its advantages in complex scenes, not only in accuracy exceeding FedAT, but also in two more complex datasets when the specified accuracy is achieved. The average is the highest than FedAT Reduce training latency by 44% and communication overhead by 36%.

In this embodiment, all clients participating in federated training are firstly grouped according to their response time, and the federated training is performed synchronously within the group to obtain a group model, and the group model is updated by asynchronous weighting between groups, and multi-agent In reinforcement learning, each client has a reinforcement learning agent, and they will decide whether to participate in the current round of model training based on their own observations. The invention can solve the problem of machine learning model training in a relatively complex heterogeneous client scene, not only can solve the long waiting delay problem in the synchronous federated learning, but also can solve the communication bottleneck problem in the completely semi-asynchronous federated learning, in addition , the present invention introduces a multi-agent reinforcement learning efficient client-side intelligent selection mode to replace the random selection strategy in the previous federated learning method, which not only optimizes the model accuracy, but also greatly improves the model training efficiency. The semi-asynchronous federated learning method spends less communication overhead and training delay when training reaches the specified accuracy, and can be applied to a wide range of heterogeneous client machine learning model training scenarios in real life.

Embodiment 2

This embodiment provides an asynchronous federated learning system based on a multi-agent model, which specifically includes the following modules:

a client intelligent selection module, which is configured to: randomly select several pre-training clients in each group of clients, and obtain the decision result of whether each pre-training client participates in the training and uploading of the model;

An intra-group synchronous training module, which is configured to: receive a local model obtained from the training of the participating models in each group of clients and the uploaded client-side training to obtain a group model;

An asynchronous training module between groups, which is configured to: perform weighted aggregation on the group model to obtain a global model.

Among them, the client intelligent selection module is specifically configured as:

Get the status of each pretrained client;

The state of each pre-training client is input to the reinforcement learning agent network, and the decision result of whether each pre-training client participates in the training and uploading of the model is obtained.

Wherein, when weighted aggregation is performed on the group models, the weight of each group model is related to the number of updates of the group model of each group.

It should be noted here that each module in this embodiment corresponds to each step in Embodiment 1 one by one, and the specific implementation process thereof is the same, which is not repeated here.

Embodiment 3

This embodiment provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the steps in the multi-agent model-based asynchronous federated learning method described in the first embodiment above .

Embodiment 4

This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and running on the processor, where the processor implements the one described in the first embodiment when the processor executes the program. Steps in an asynchronous federated learning approach based on a multi-agent model.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the invention may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media having computer-usable program code embodied therein, including but not limited to disk storage, optical storage, and the like.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each process and/or block in the flowchart illustrations and/or block diagrams, and combinations of processes and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. During execution, the processes of the embodiments of the above-mentioned methods may be included. The storage medium may be a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM), or a random access memory (Random Access Memory, RAM) or the like.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims (10)

1. an asynchronous federated learning method based on a multi-agent model is characterized in that, comprising: Randomly select several pre-training clients in each group of clients, and obtain the decision result of whether each pre-training client participates in model training and uploading; Receive the local model obtained from the training of the model and the uploaded client in each group of clients, and obtain the group model; A weighted aggregation of the group models results in a global model. 2. a kind of asynchronous federated learning method based on multi-agent model as claimed in claim 1 is characterized in that, the acquisition method of described decision result is: Get the status of each pretrained client; The state of each pre-training client is input to the reinforcement learning agent network, and the decision result of whether each pre-training client participates in the training and uploading of the model is obtained. 3. The asynchronous federated learning method based on a multi-agent model according to claim 2, wherein the state of the pre-training client comprises: the training round index t, the data size on the client, the The number of times the client participated in local model updates and uploads until the t round, the number of updates of the group model the client was in until the t round, the communication overhead of all pre-trained clients, and the training delay of all pre-trained clients. 4 . The asynchronous federated learning method based on a multi-agent model according to claim 2 , wherein the reinforcement learning agent network aims at maximizing cumulative returns. 5 . 5. A kind of asynchronous federated learning method based on multi-agent model as claimed in claim 1, is characterized in that, when group model is weighted and aggregated, the weight of each group model is related to the update times of the group model of each group . 6. An asynchronous federated learning system based on a multi-agent model, comprising: a client intelligent selection module, which is configured to: randomly select several pre-training clients in each group of clients, and obtain the decision result of whether each pre-training client participates in the training and uploading of the model; An intra-group synchronous training module, which is configured to: receive a local model obtained from the training of the participating models in each group of clients and the uploaded client-side training to obtain a group model; An asynchronous training module between groups, which is configured to: perform weighted aggregation on the group model to obtain a global model. 7. A kind of asynchronous federated learning system based on multi-agent model as claimed in claim 6, it is characterised in that the client intelligent selection module is specifically configured as: Get the status of each pretrained client; The state of each pre-training client is input to the reinforcement learning agent network, and the decision result of whether each pre-training client participates in the training and uploading of the model is obtained. 8. A multi-agent model-based asynchronous federated learning system as claimed in claim 6, characterized in that, when the group model is weighted and aggregated, the weight of each group model is related to the number of updates of the group model of each group . 9. A computer-readable storage medium on which a computer program is stored, characterized in that, when the program is executed by a processor, a multi-agent model-based asynchronous system as claimed in any one of claims 1-5 is implemented Steps in a federated learning approach. 10. A computer device, comprising a memory, a processor and a computer program stored on the memory and running on the processor, wherein the processor implements any of claims 1-5 when the processor executes the program. Steps in an asynchronous federated learning method based on a multi-agent model described in a paper.

Images