CN114819181A - Multi-objective federated learning evolution method based on improved NSGA-III - Google Patents

Abstract

The invention belongs to the field of artificial intelligence, and discloses a multi-objective federated learning evolution method based on improved NSGA-III, comprising the following steps: acquiring learning data; constructing a federated learning multi-objective optimization model; fast greedy initialization and multi-objective evaluation; Dominant sorting; perform iteration, each iteration performs the following operations: selection, crossover, mutation operator operations, generating Q _t , performing federated learning training evaluation on the offspring population, and calculating the three goals of each individual; mixing parent Generation and offspring populations R _t =Q _t +P _t ; sort R _t non-dominantly, and select P _t+1 based on the reference point; find out the Pareto optimal solution, and output the label corresponding to the Pareto optimal solution result. Compared with the classical algorithm, the present invention can obtain a better Pareto solution, and under the condition of ensuring the accuracy of the global model, the communication cost and the distribution variance of the accuracy of the global model are reduced.

Description

Multi-objective federated learning evolutionary method based on improved NSGA-III

technical field

The invention belongs to the technical field of artificial intelligence, and particularly relates to a multi-objective federated learning evolution method based on improved NSGA-III.

Background technique

The rapid development of artificial intelligence brings great convenience to society, but also brings some hidden dangers, such as data silos and privacy leakage. Traditional centralized machine learning needs to gather scattered data together for machine learning training, but in fact it is difficult to aggregate data in many fields. For example, it is difficult to share data between hospitals, and there is a serious problem of "data silos". . In addition, the problem of privacy leakage has emerged, people's awareness of privacy protection has gradually increased, and countries around the world have also introduced privacy protection laws and regulations.

Therefore, federated learning emerges as a viable solution to the problems of data silos and privacy leaks. Federated learning can train a good global model while keeping the data local to the participants. After each participant downloads the current global model from the server, trains it in the local data, and then uploads the trained local model to the server for aggregated update of the model. After multiple rounds of iterations, a global model with good performance is finally obtained. .

However, traditional federated learning still faces the challenges of high communication cost and structural heterogeneity. The parameter transmission between the federated learning server and the participants consumes a lot of communication costs; at the same time, due to different computing and storage capabilities and different network environments between different participants, the participants will be offline during training, and the parameters of the transmission model will be lost, etc. The efficiency, accuracy, and fairness of federated learning are affected. There is currently much literature devoted to the study of communication cost or structural heterogeneity, but less comprehensive consideration of these issues.

In view of the above problems, the present invention comprehensively considers the problem of communication cost and structural heterogeneity, and studies the balance between model validity, fairness and communication cost. The present invention first defines federated learning as a three-objective optimization model, aiming at maximizing the global model accuracy rate, minimizing the global model accuracy rate distribution variance and communication cost at the same time. Based on the training characteristics of federated learning, the initialization of the third-generation non-dominated sorted genetic algorithm-III (NSGA-III) is improved, and a multi-objective federated learning based fast greedy algorithm is designed. The initialized NSGA-III algorithm (Fast greedy initialization NSGA-III, FNSGA-III). The experimental results show that the FNSGA-III algorithm can achieve the balance of the three objectives, and can effectively reduce the communication cost and the variance of the accuracy distribution of each participant without seriously losing the overall performance of the FL model. The accuracy distribution is more balanced. The work done by the present invention mainly includes:

(1) According to the author's current knowledge, the present invention comprehensively considers the objectives of maximizing the accuracy of the global model, minimizing the distribution variance of the accuracy of the global model, and minimizing the communication cost, and constructs a federated learning multi-objective optimization model for the first time.

(2) Propose FNSGA-III algorithm. In order to quickly converge the NSGA-III algorithm and obtain high-quality solutions, it is more convenient to solve the multi-objective optimization model of federated learning, an initial solution construction algorithm based on fast greedy initialization is proposed, and binary and real-valued encoding and decoding strategies are introduced to speed up NSGA-III. Algorithm evolution efficiency.

(3) Through the MNIST data set experiment, it is verified that the Pareto solution obtained by the FNSGA-III algorithm is better than the NSGA-III algorithm. The FNSGA-III algorithm's hyper-volume HV index value can reach 127.55% of the NSGA-III algorithm, and the running time is optimal. The case is 73.96% of the NSGA-III algorithm. In addition, comparing the FNSGA-III algorithm with other classical evolutionary algorithms NSGA-II and SPEA2, the results show that the Pareto solution quality obtained by the FNSGA-III algorithm is higher. Finally, some Pareto solutions are selected for federated learning experiments. The Pareto solutions obtained by the algorithm can effectively reduce the communication cost and the distribution variance of the global model accuracy while ensuring the accuracy of the global model.

In recent years, federated learning has received extensive attention. McMahan first proposed the concept of federated learning and Federated Averaging (FedAvg) in 2016, which has important application significance for privacy protection and machine learning training under data silos. The research of federated learning continues to deepen, but it still has some unsurmountable challenges, such as high communication cost and structural heterogeneity.

In order to make federated learning applicable to massive data, we must consider reducing the communication overhead of federated learning. The FedAvg algorithm proposed by McMahan et al. reduces the number of global communication rounds by increasing the local training computation in each round of communication, thereby improving communication efficiency. Some scholars reduce the amount of communication transmission by reducing the size of the parameters uploaded by the participants. Chen et al. proposed a layered asynchronous update algorithm. The authors layered the parameters into shallow parameters and deep parameters according to the structural characteristics of the deep neural network model. In the early global communication iteration process, between local participants and the server Only the shallow parameters are transmitted, and the deep parameters of the global model are only transmitted and aggregated in the last few rounds of communication. The algorithm reduces the communication overhead by reducing the size of the parameters of the transmission model and the update frequency of the deep parameters in the neural network. However, the accuracy of the model will be affected. Zhu et al. introduced the sparse evolutionary training (SET) algorithm into federated learning. The main idea of the SET algorithm is to control the connection sparseness between the fully connected networks through the sparsity parameter between the fully connected layers of the neural network. In order to reduce the size of the parameters of the transmission model, the communication cost is effectively reduced.

Besides communication cost, structural heterogeneity is also one of the main problems for federated learning optimization. Due to different computing and storage capabilities and different network environments among different participants, the participants will be offline during training, and the parameters of the transmission model will be lost. In order to enhance the robustness of federated learning, some scholars have conducted various studies on the problem of structural heterogeneity. Hao et al. designed a secure aggregation protocol that allows participants to withdraw at any time, as long as the number of remaining participants can satisfy the federated learning update, which improves the fault tolerance and robustness of the system. Some scholars have studied how to reasonably allocate heterogeneous equipment resources. Kang et al. considered the cost difference of participants to motivate more high-quality participants to perform federated learning training. Using the global model accuracy variance as a fairness measure, Li et al. designed a q-FFL (q-Fair federated learning) optimization algorithm, which increased the model aggregation weight of high-loss participants. The accuracy of low-accuracy participants is improved, the performance distribution between different participants is balanced, and the fair resource allocation of federated learning is promoted.

The above research is carried out on one aspect of communication cost or structural heterogeneity, and the optimization of the federated learning algorithm has been achieved to different degrees and different goals. There are requirements for costs, etc. In order to achieve the balance of multiple goals under the federated learning framework, some scholars have tried to combine intelligent optimization algorithms with federated learning. Zhu et al. defined federated learning as a dual-objective optimization problem, aiming to minimize the model test error rate and communication cost, and used the NSGA-II (non-dominated sorted genetic algorithm-II) algorithm to optimize the neural network structure parameters of federated learning. Compared with the standard FedAvg algorithm, the Pareto solution evolved by the algorithm can simultaneously improve the model performance and communication efficiency to a certain extent, but the algorithm does not consider the communication instability and participants caused by the structural heterogeneity of federated learning. In other cases, the accurate distribution is not balanced, and the NSGA-II algorithm used at the same time has poor scalability for the multi-objective federated learning model. Basheer et al. used particle swarm optimization to update the number of hidden layers, neurons and global communication rounds of the neural network optimized for federated learning, but the optimization objective was a single objective, and other objectives of federated learning were not considered comprehensively.

SUMMARY OF THE INVENTION

In view of the above problems, the present invention comprehensively considers the communication cost and structural heterogeneity, and introduces fairness as an optimization objective to explore the multi-objective equilibrium relationship between model accuracy, fairness and communication cost of federated learning. And the communication environment is set to be unstable in the experiment to enhance the robustness of the algorithm.

The multi-objective federated learning evolution method based on the improved NSGA-III proposed by the present invention is applied to the server and multiple participants, including the following steps:

acquiring learning data, the learning data is used for labeling;

Constructing a federated learning multi-objective optimization model, the multi-objective optimization model includes three objectives of maximizing the accuracy of the global model, minimizing the distribution variance of the accuracy of the global model, and communication cost;

Fast greedy initialization P _t = 0, multi-objective evaluation using FedAvg algorithm;

Perform non-dominant sorting on P _t ;

Iteration is performed, and the following operations are performed in each iteration: selection, crossover, and mutation operator operations to generate Q _t , the training evaluation of federated learning is performed on the offspring population, and the three goals of each individual are obtained by calculation; the parent and offspring populations are mixed. R _t =Q _t +P _t ; sort R _t non-dominantly, and select P _t+1 depending on the reference point;

Find the Pareto optimal solution and output the labeling result corresponding to the Pareto optimal solution.

Further, in the evolutionary process of federated learning, the loss function of the _kth participant with the dataset Dk is:

The global goal of federated learning evolutionary methods is to minimize the global loss function L(w) as follows:

where k is the serial number of the participant, L _k (w) is the loss function of the kth participant, l _i (w) is the loss function on the data sample i, and n _k is the size of the dataset D _k of participant k n _k = |D _k |, where n is the total data sample size of K participants.

然后参与者将更新后的本地模型发送给服务器，服务器以一定的规则 聚合各个模型，得到一个新的全局模型w_t+1，用于下一轮次的迭代训练，下标 t表示联邦学习的通信轮次。Further, in each round of training in the federated learning process, each participant receives the global model _wt from the server, and uses the local data to train the global model to obtain an updated local local model.

Then the participant sends the updated local model to the server, and the server aggregates each model according to certain rules to obtain a new global model w _t+1 for the next round of iterative training, where the subscript t represents the federated learning Communication rounds.

Further, the three-objective optimization model of federated learning evolution is:

Among them, F(v) is the objective function of the three-objective optimization model. This model has three minimization objectives, which are to minimize the global model test error rate f ₁ , the global model accuracy distribution variance f ₂ , and the communication cost f ₃ , Conv is the number of convolution layers, kc is the number of convolution kernels, ks is the size of the convolution kernels, L is the number of fully connected layers, N is the number of neurons in the fully connected layer, η is the learning rate, and ε is the connectivity parameter of the neural network , the number of connections between the two fully connected layers is determined by the connectivity parameter ε, and the total number of connections is n=ε(n ^k +n ^k-1 ), where n ^k and n ^k-1 are the k layers and k respectively - Number of neurons in layer 1.

{a₁,a₂,...,a_K}为各个参与者的准确率。Further, the global model test error rate E=1-A of the target f ₁ , A is the average test accuracy rate of the global model

{a ₁ ,a ₂ ,...,a _K } is the accuracy of each participant.

Further, the target f ₂ is the global model accuracy distribution variance

K是参与者总数，C是 参与者每轮参与比例，σ是模型参数大小。Further, the target _f3 can be expressed as

K is the total number of participants, C is the proportion of participants in each round, and σ is the size of the model parameters.

Further, the described process of carrying out the training evaluation of federated learning to the offspring population is realized by the FedAvg algorithm based on static SET, and specifically includes the following steps:

i is an individual of the population in the FNSGA-III algorithm, P is the population size, after decoding the individual i, the relevant federated learning neural network hyperparameters, the connectivity of the neural network and the participation ratio C _i of the participants in each round are obtained;

Use the communication parameter ε _i to initialize the static SET topology, and use the static SET topology as the global model used in the algorithm;

During each round of training, use the mini-batch stochastic gradient descent method to train the local data;

After a certain number of rounds, the test error rate of the global model, the distribution variance of the global model accuracy rate and the communication cost are calculated.

Further, the number of convolutional layers, the number of convolutional kernels, the size of convolutional kernels, the number of fully connected layers, the number of neurons in each layer of the fully connected layer, and the SET parameter ε of MLP and CNN use binary encoding, the learning rate η and the participation in each round Scale C uses real-valued encoding.

Further, the learning data is the MNIST data set.

The beneficial effects of the present invention are as follows:

The present invention proposes FNSGA-III algorithm to solve the problem of multi-objective federated learning model, and conducts experimental verification under unstable communication conditions. We first constructed a three-objective template for federated learning, and set the optimization objective to minimize the global model test error rate, communication cost, and global model accuracy distribution variance. The decision variables were the hyperparameters of the neural network and the federated learning parameters. The NSGA-III algorithm is introduced to solve the federated learning multi-objective model, and the initialization of NSGA-III is changed. The experimental results show that the improved FNSGA-III algorithm is better than the original NSGA-III algorithm. And compared with the benchmark federated average algorithm, the Pareto optimal solution optimized by the FNSGA-III algorithm effectively improves the accuracy of the global model and reduces the distribution variance and communication cost of the global model accuracy.

Description of drawings

Figure 1. The federated learning training process diagram;

Fig. 2 chromosome coding example diagram of MLP model of the present invention;

Fig. 3 chromosome coding example diagram of CNN model of the present invention;

Fig. 4 algorithm flow chart of the present invention;

Fig. 5 is an experiment contrast diagram on IID of the MLP of the present invention and NSGA-III algorithm;

Fig. 6 is an experiment comparison diagram of the present invention and the MLP of NSGA-III algorithm on non-IID;

Fig. 7 is the experiment contrast diagram of the CNN of the present invention and NSGA-III algorithm on IID;

Fig. 8 is the experiment comparison diagram of the present invention and the CNN of NSGA-III algorithm on non-IID;

Figure 9 Pareto optimal solution diagram of the present invention, NSGA-II and SPEA2.

Detailed ways

The present invention is further described below in conjunction with the accompanying drawings, but the present invention is not limited in any way, and any transformation or replacement based on the teaching of the present invention belongs to the protection scope of the present invention.

In order to achieve this purpose, the technical scheme adopted in the present invention comprises the steps as follows:

In order to make the technical solutions and beneficial effects of the present invention clearer, the present invention will be further described below with reference to practical examples. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

Example

Federated learning is a privacy-preserving machine learning technique that enables participants to jointly train a global model without uploading local private data to the server. Suppose there are K participants with data {D ₁ ,D ₂ ,...,D _K }, the traditional centralized learning is to put all the data together, using D=D ₁ ∪ D ₂ ... D _K to Train the model.

In the federated learning process, the loss function of the _kth participant with the dataset Dk is:

The global goal of federated learning is to minimize the global loss function L(w):

In formulas (1) and (2), k is the serial number of the participant, L _k (w) is the loss function of the kth participant, l _i (w) is the loss function on the data sample i, and n _k is the participation function. The size of the dataset D _k of participant k is n _k = |D _k |, where n is the total size of the data samples of K participants. The goal of federated learning is to optimize the global loss function L(w) by minimizing the weighted average of the participant loss functions _Lk (w). Federated learning is a collaborative process, as shown in Figure 1.

然后参与者将更新后 的本地模型发送给服务器，服务器以一定的规则聚合各个模型，得到一个新 的全局模型w_t+1，用于下一轮次的迭代训练。下标t表示联邦学习的通信轮次。 第3代非支配排序遗传算法(NSGA-III)In each round of training, each participant receives the global model _wt from the server, and uses the local data to train the global model to obtain an updated local local model

Then the participant sends the updated local model to the server, and the server aggregates each model according to certain rules to obtain a new global model w _t+1 for the next round of iterative training. The subscript t denotes the communication round of federated learning. The third generation of non-dominated sorting genetic algorithm (NSGA-III)

There are many research results of multi-objective optimization evolutionary algorithm based on genetic algorithm and Pareto optimal solution, such as the second generation non-dominated sorting genetic algorithm (NSGA-II), multi-objective evolutionary algorithm based on decomposition (Multi-objective evolutionary algorithm based on decomposition) , MOEA/D), SPEA2 (Strength ParetoEvolutionary Algorithm 2), PAESPareto archived evolution strategy). NSGA-II is a powerful and robust multi-objective evolutionary algorithm suitable for problems with two or three objectives. If the number of objectives is greater than 3, newer evolutionary algorithms can be used, such as the 3rd generation non-dominated sorting genetic algorithm based on reference points (NSGA-III), NSGA-III is used for optimization problems with four or more objectives The performance is better than NSGA--II. The present invention defines federated learning as a three-objective optimization problem model. In order to ensure the scalability of the algorithm objectives, for example, when the objectives of the federated learning are expanded to four or more, the algorithm is still applicable. The present invention adopts the NSGA-III algorithm, which The basic steps can be outlined as:

Step 1: Initialize the reference point and the parent population P _t (population size is N), and perform non-dominated sorting of individuals in the population, normalize individuals, and associate to the reference point.

Step 2 uses selection, crossover, and mutation operators on P _t to create a child population Q _{t with the same size as the parent population P t .}

Step 3 mixes P _t and Q _t into a new population R _t , where R _t population size is 2N. The merged population is sorted non-dominated, divided into different non-dominated solution sets (F ₁ , F ₂ ,..., F _s ), and individual normalization is performed to associate with reference points.

Step 4: Select N solutions from the sorted population R _t to generate the next generation parent population P _t+1 . If the number of selected non-dominated frontier solutions is greater than N, in the non-dominated solution set of the last layer to be selected, a reference point-based selection method is used to select solutions.

Step 5, go to Step 2, repeat the whole process until the preset stopping conditions are met, and output the Pareto optimal solution.

The present invention builds a three-objective optimization model of federated learning on the basis of a typical multi-objective optimization model, and elaborates its objectives, decision variables, variable codes, and the like. The three-objective optimization model of the federated learning of the present invention is:

(1) Objective function:

F(v) is the objective function of the model. This model has three minimization objectives, which are to minimize the global model test error rate f ₁ , the global model accuracy distribution variance f ₂ , and the communication cost f ₃ .

由此，可以计算出目标f₁全局模型测试错误率E＝1-A。In the present invention, the specific evaluation process of the three goals of federated learning is to use the FedAvg algorithm combined with the SET algorithm to conduct training for a certain round of communication, test the trained global model w, and obtain the accuracy rate {a ₁ , a ₂ ,...,a _K }. Calculate the average test accuracy of the global model

Thus, the target f ₁ global model test error rate E=1-A can be calculated.

方差可以看作公平性 的度量指标之一，将公平性作为优化目标，可以尽量避免出现平均精度高， 但单个参与者没有准确率保证的情况。通过减小参与方之间全局模型准确率 分布的方差，从而使聚合后参与者准确率分布更均匀、更公平。The target f ₂ is the variance of the global model accuracy distribution

Variance can be regarded as one of the measures of fairness. Taking fairness as the optimization goal can try to avoid the situation where the average precision is high, but there is no guarantee of accuracy for a single participant. By reducing the variance of the global model accuracy distribution among the participants, the accuracy distribution of the participants after aggregation is more uniform and fair.

K是参与者总数，C是参与者每轮参与比 例，σ是模型参数大小。The target _f3 is the average communication cost of the participants. It is assumed that the communication cost of each participant is only related to the size of the model parameters it transmits. Since the neural network structure used in the present invention does not change during the federated learning training process, each The model parameter size σ of the participants is the same and remains unchanged, then the target _f3 can be expressed as

K is the total number of participants, C is the proportion of participants in each round, and σ is the size of the model parameters.

(2) Model decision variables and their constraints

Since federated learning is a process of collaborative training of machine learning models, in the present invention, decision variables and constraints are parameters to be optimized and their ranges, which are represented by v. The parameters we set for optimization include the neural network hyperparameters of federated learning, the connectivity parameter ε of the neural network, and the participation ratio C in each round.

The neural network of the present invention selects a multi-layer perceptron (MLP) and a convolutional neural network (CNN), wherein the hyperparameters under the MLP model include the number of hidden layers L, the number of neurons in each hidden layer Number N, learning rate η; CNN hyperparameters include the number of convolution layers Conv, the number of convolution kernels kc, the size of the convolution kernel ks, the number of fully connected layers L, the number of fully connected layer neurons N, and the learning rate η. That is, v={Conv,kc,ks,L,N,ε,η,C}, the value range of each variable is set in the experimental part.

The connectivity parameter ε of the neural network draws on the static SET algorithm in the SET algorithm proposed by Mocanu, that is, firstly, the sparse weight matrix is initialized with an ER random graph between the two fully connected layers, and then the topology of the network is kept fixed. The number of connections between the two layers is determined by the parameter ε, and the total number of connections is n=ε(n ^k +n ^k-1 ), where n ^k and n ^k-1 are neurons in layers k and k-1 respectively number. In the present invention, the static SET algorithm is used in the fully connected layers of MLP and CNN.

(3) Coding of decision variables

We adopt the FNSGA-III algorithm to optimize the neural network hyperparameters, connectivity parameter ε and participation ratio C of each round of federated learning. Chromosome is the main body of algorithm operation. In the present invention, we have two types of decision variables, integer and real numbers, which need to be coded. All integers use binary coding, and real numbers use real-value coding. Then the number of convolutional layers, the number of convolutional kernels, the size of convolutional kernels, the number of fully connected layers, the number of neurons in each layer of the fully connected layer and the SET parameter ε of MLP and CNN use binary coding, the learning rate η and the participation ratio C in each round Use real-value encoding. Encoding examples of MLP and CNN are shown in Figure 2 and Figure 3.

During the decoding process, 1 is automatically added during binary decoding. For example, 000000 is decoded as 1. In the MLP example, N ₁ is encoded as 000111 and decoded as N ₁ =8. For convenience, the size of the convolution kernel in CNN is only selected between 3 and 5, and the convolution output is always kept unchanged. In the neural network structure of CNN, only a pooling layer is added at the end.

In order to speed up the search performance, the present invention improves the random initialization in NSGA-III to be a fast greedy initialization suitable for federated learning, and records the improved NSGA-III algorithm as the FNSGA-III algorithm. Among them, the fast greedy initialization process is briefly described as follows:

(1) Randomly generate the initial solution of l-ploid population;

(2) After randomly dividing all participants into groups of the same size, the federated learning training and evaluation process of the initial solution is performed synchronously in each group. Among them, the number of participants in each round of federated learning, the number of local training rounds, and the number of global communication rounds are all reduced, which can quickly obtain the three goals after federated learning training, and complete the evaluation of all initial solutions;

(3) Then select the optimal doubling population solutions for the three objectives respectively;

(4) After mixing and removing duplicate solutions, randomly select the solution with the specified population number.

FNSGA-III algorithm flow

For the three-objective optimization model of federated learning, the present invention uses the FNSGA-III algorithm to solve the problem to obtain a set of Pareto optimal solutions. The algorithm flowchart is shown in FIG. 4 .

FNSGA-III first uses fast greedy initialization to generate an initial population of size N, that is, the first-generation parent population, and encodes the corresponding variables as binary and real-valued chromosomes. The iterative process mainly uses binary tournaments to select two parent individuals to generate two offspring individuals. The crossover mutation algorithm uses single-point crossover and flip mutation on binary chromosomes, and simulated binary crossover on real-valued chromosomes ( Simulated binary crossover, SBX) and polynomial variation. Repeat this process until N offspring individuals are produced.

Then, the training evaluation of federated learning is performed on the offspring population, and the three goals of each individual are calculated. The parent and child populations are mixed, and the mixed population is sorted non-dominantly. N individuals are selected as the parent population of the next generation. Repeat these steps until the iteration stop condition is met. Finally, a set of Pareto optimal solutions are obtained and analyzed in depth.

The specific evaluation process of federated learning is based on the FedAvg algorithm combined with the static SET algorithm. The pseudo-code of the federated learning evaluation process under the FNSGA-III algorithm is shown in Algorithm 1.

In Algorithm 1, i is an individual of the population in the FNSGA-III algorithm, and P is the population size. After decoding the individual i, the relevant federated learning neural network hyperparameters, the connectivity of the neural network and the participation ratio C _i of each round of participants are obtained. First, the static SET topology is initialized with the connectivity parameter _εi as the global model used in the algorithm; during each round of training, the local data is trained using Mini-batch SGD. After a certain number of rounds, the test error rate of the global model, the distribution variance of the accuracy rate of the global model, and the communication cost are calculated.

In this section, we will introduce the experimental setup of the present invention. It mainly includes the following parts: (1) experimental environment and experimental data set; (2) neural network related parameters and sparse connectivity parameters used in the experiment; (3) federated learning parameters and data division methods; (4) FNSGA -III parameter.

The experimental environment is an Ubuntu system based on Intel(R) Core(TM) i9-9900KF CPU@3.60GHz×16. Each experiment is trained and tested on the MNIST dataset, which consists of 28 × 28 pixel images of handwritten digits, with 60 000 training images and 10 000 testing images.

We chose MLP and CNN as the neural network models for federated learning training, and empirically set the standard MLP and CNN parameters of the present invention. Among them, there are 2 hidden layers in the MLP, each layer has 200 neurons (the parameter amount is 199210), and the ReLu function is used as the activation function. The CNN model sets two 5×5 convolutional layers (the first has 32 channels and the second has 64 channels), followed by a 2×2 Max pooling layer, a fully connected layer of 128 neurons, The ReLu activation function is used, and finally a 10-class softmax output layer (with a parameter size of 1 659 146). In MLP and CNN, the learning rate η of the Mini-batch SGD algorithm is 0.05 and the batch size B is 10. The static SET algorithm is used in the fully connected layer of MLP and CNN, and the present invention sets the network sparsity parameter ε=20. The above parameter settings are used as the standard neural network structure in the experiments of the present invention.

In federated learning, we set the total number of participants K as 100, and the participant participation ratio C as 1, that is, there are 100 × 1 participants in each round of communication. For participant local model training, the iteration epoch is set to 5. Since the size and distribution of data between different participants are often different, we study the following two real-world scenarios: the first independent identical distribution (IID), first shuffle the MNIST data, then 100 Each of the participants contains 600 samples; the second non-Independent Identically Distribution (non-IID), first sorts the marked digital labels, divides them into 200 segments on average according to 300 samples, and gives 100 Each participant is assigned two segments, each participant contains only two labels and the data sample size is the same. Because the present invention assumes that the communication environment of federated learning is unstable and the model parameter transmission is lost, the loss rate Drop=30% is set here.

Next, we set the parameters for FNSGA-III, where the population size was set to 20, and 20 iterations were performed. The selection operator adopts the two-round tournament, the single-point crossover with probability 0.9 is used in the binary chromosome, and the bit-flip mutation with probability 0.1 is used; the simulated binary crossover with probability 0.9 and _nc = 2 is used in the real-valued chromosome, and the sum probability is Polynomial variation of 0.1 and _nm = 20.

The present invention uses the FNSGA-III algorithm to optimize the relevant parameters of federated learning, so as to realize the difference between the global model test error rate, the global model accuracy rate distribution variance and the communication cost in the federated learning. We choose MLP and CNN as the neural network for federated learning training. model, and empirically set the standard MLP and CNN parameters of the present invention. Among them, there are 2 hidden layers in MLP, each layer has 200 neurons (parameters are 199 210), and the ReLu function is used as the activation function. The CNN model sets two 5×5 convolutional layers (the first has 32 channels, the second has 64 channels), followed by a 2×2 Max pooling layer, a fully connected layer of 128 neurons, Using the ReLu activation function, finally a 10-class softmax output layer (with a parameter size of 1 659 146). In MLP and CNN, the learning rate η of the Mini-batch SGD algorithm is 0.05 and the batch size B is 10. The static SET algorithm is used in the fully connected layer of MLP and CNN, and the present invention sets the network sparsity parameter ε=20. The above parameter settings are used as the standard neural network structure in the experiments of the present invention.

In federated learning, we set the total number of participants K as 100, and the participant participation ratio C as 1, that is, there are 100 × 1 participants in each round of communication. For participant local model training, the iteration epoch is set to 5. Since the size and distribution of data between different participants are often different, we study the following two real-world scenarios: the first independent identical distribution (IID), first shuffle the MNIST data, then 100 Each of the participants contains 600 samples; the second non-Independent Identically Distribution (non-IID), first sorts the marked digital labels, divides them into 200 segments on average according to 300 samples, and gives 100 Each participant is assigned two segments, each participant contains only two labels and the data sample size is the same. Because the present invention assumes that the communication environment of federated learning is unstable and the model parameter transmission is lost, the loss rate Drop=30% is set here.

Next, we set the parameters for FNSGA-III, where the population size was set to 20, and 20 iterations were performed. The selection operator adopts the two-round tournament, the single-point crossover with probability 0.9 is used in the binary chromosome, and the bit-flip mutation with probability 0.1 is used; the simulated binary crossover with probability 0.9 and _nc = 2 is used in the real-valued chromosome, and the sum probability is Polynomial variation of 0.1 and _nm = 20.

The present invention uses the FNSGA-III algorithm to optimize the relevant parameters of the federated learning, so as to achieve a balance between the global model test error rate, the global model accuracy rate distribution variance and the communication cost in the federated learning. Firstly, a comparative experiment is carried out on the FNSGA-III algorithm and the NSGA-III algorithm to explore the effectiveness of the proposed method. The relevant parameter settings for the experimental analysis of the multi-objective evolutionary algorithm are shown in the figure.

Table 1 Multi-objective federated learning evolutionary algorithm related parameter settings

parameter MLP CNN population size 20 20 The maximum number of iterations 20 20 Federated Learning Participation Rate 0.1-1 0.1-1 learning rate 0.01-0.2 0.01-0.2 SET parameter 1-128 1-128 MLP hidden layers 1-4 The number of neurons in the hidden layer of MLP 1-256 CNN convolutional layers 1-3 CNN convolution kernel channel number 1-64 CNN convolution kernel size 3 or 5 CNN fully connected layers 1-3 The number of neurons in the fully connected layer of CNN 1-256

We set the population size to 20, the population iteration generation to 20, and the number of communication rounds for each individual to perform the federated learning evaluation process to 5. Set the value range of the participant participation ratio parameter C from 0.1 to 1, to ensure that a certain number of participants participate in the training, and the learning rate is 0.01 to 0.2. Too high a learning rate will affect the convergence.

In the parameter setting of the neural network, the maximum number of hidden layers of MLP is 4, and the maximum number of neurons in each layer is 256. For CNN, set the maximum number of convolutional layers to 3, the maximum number of kernel channels to 64, the maximum number of fully connected layers to 3, the maximum number of neurons in the convolutional layer to be 256, and the size of the convolution kernel to be 3 or 5. The maximum value of the network sparsity parameter is set to 128.

The final Pareto solutions of MLP and CNN using FNSGA-III algorithm and NSGA-III algorithm evolution on IID and non-IID data are shown in Figure 5-Figure 8, where each point represents the corresponding structure parameter in federated learning. a solution. The light-colored points represent the Pareto optimal solution obtained under the FNSGA-III algorithm of the present invention, and the dark-colored points represent the Pareto optimal solution obtained under the randomly initialized NSGA-III algorithm.

The number of Pareto solutions obtained by FNSGA-III is more stable than that obtained by NSGA-III, and the number of Pareto solutions of NSGA-III under CNN is less, for example, the number of Pareto solutions of NSGA-III under CNN IID is only 3. In Fig. 5-Fig. 8, the light-colored solution is superior to the dark-colored solution, that is, the Pareto solution obtained by the present invention is superior to the Pareto solution of the NSGA-III algorithm, except that under CNN IID, a dark-colored solution is superior to the light-colored solution. color solution, but the number of solutions under CNN IID is very small. And the FNSGA-III algorithm under IID has a more dominant effect. The distance between the solutions of FNSGA-III and NSGA-III is quite different, while the distance between the solutions of the two algorithms under non-IID is smaller. At the same time, we find that FNSGA-III will converge more at the inflection point. The solution at the inflection point is characterized by small target values and higher solution quality. The Pareto solution of FNSGA-III is concentrated at the inflection point, which makes its uniformity under MLP inferior to Homogeneity of the Pareto solution of NSGA-III.

In addition, Table 2 shows the related evaluation index results of the Pareto solutions finally obtained by FNSGA-III and NSGA-III. The Pareto solutions of the two algorithms are analyzed from multiple dimensions.

The minimum value of a single objective reflects the extreme value of each objective function and reflects the optimization ability of the algorithm. It can be seen from Table 2 that the minimum global model test error rate, minimum variance, and minimum communication cost obtained by FNSGA-III under a single target are basically smaller than NSGA-III. For example, in terms of the minimum global model test error rate, FNSGA- The value of III is smaller than that of NSGA-III, and the difference is obvious under CNN non-IID, with a difference of 4.89%.

The number of Pareto non-dominated solutions obtained by the FNSGA-III algorithm is stable, except for MLP IID, the other numbers are larger than those of NSGA-III. On average, the number of solutions obtained by the two algorithms under MLP is larger than that under CNN. The number of MLP solutions of NSGA-III is significantly larger than that of CNN, and the change is obvious. Then the FNSGA-III algorithm is more robust than the NSGA-III algorithm in terms of the number of solutions.

The comprehensive index Hypervolume index (Hypervolume, HV), which calculates the sum of the hypervolume of the hypercube composed of all non-dominated solutions and reference points, is a comprehensive index for evaluating the Pareto solution. The quality of the evaluated Pareto solution is better. It can be seen from Table 2 that the HV value of the FNSGA-III algorithm is always better than that of NSGA-III, showing better quality.

Table 2 Analysis of each index of FNSGA-III algorithm and NSGA-III algorithm

The coverage ratio C(A, B) calculates the proportion of solutions in solution set B dominated by at least one solution in A, and measures the degree of overlap between the two solution sets. The larger the index C, the more the solution set A is. The quality of is better than the quality of solution set B. In Table 2, F in C(F,N) represents the solution set of FNSGA-III, and N represents the solution set of NSGA-III. The metric value of C(F,N) is basically greater than that of C(N,F), Under CNN IID, C(F,N)=50% is smaller than C(N,F)=53%, but the difference between the two is very small. Overall, the solution of FNSGA-III is better than the NSGA-III algorithm in terms of coverage C.

In terms of running time, the time consumption of CNN is greater than that of MLP, the time consumption of non-IID is greater than that of IID, and the time consumption of NSGA-III is greater than that of FNSGA-III, that is, from the statistical results, FNSGA-III has the best time performance. According to the above analysis of single objective minimum value, number of non-dominated solutions, HV index, C index and time performance, we can conclude that the Pareto optimal solution obtained by the proposed FNSGA-III algorithm has better quality than NSGA-III algorithm solution.

In addition, we compare the proposed algorithm FNSGA-III with NSGA-II and SPEA2 evolutionary multi-objective algorithm, and evaluate the results of the algorithm in terms of HV comprehensive index, number of Pareto solutions, coverage C, time, and single-objective optimal solution. The pros and cons of Pareto optimal solutions. In the above experiments, MLPnon-IID, which is the closest to the Pareto obtained by FNSGA-III and NSGA-III, is selected to carry out comparative experiments of multiple evolutionary algorithms. The experimental results are shown in Table 3 and Figure 6.

Table 3 Analysis of each index of the experiment of FNSGA-III algorithm and other evolutionary algorithms under MLP non-IID

A simple analysis of Table 3 shows that the Pareto solution of FNSGA-III is more concentrated at the inflection point and is superior to the Pareto solution of NSGA-II and SPEA2, that is, each of the three objectives is better than the NSGA-II and SPEA2 algorithms. Coverage is all compared with FNSGA-III. The HV value of FNSGA-III, the number of Pareto solutions, the coverage ratio, and the minimum value under the three objectives are all better than those of NSGA-II and SPEA2. In terms of time, the running time of SPEA2 is the shortest, but the running time of FNSGA-III is not much different from that of SPEA2. In summary, the proposed FNSGA-III algorithm changes the random initialization to fast greedy initialization, which can improve the operation efficiency and is basically better than the NSGA-III, NSGA-II, SPEA2 evolutionary algorithms, and the obtained Pareto has higher quality.

Due to the very small set of communication rounds in the federated learning evaluation process of FNSGA-III, its federated learning performance has not been fully explored. Due to the limitation of computational resources, we only select the MLP non-IID with the worst accuracy for augmentation experiments. Four solutions are selected for the Pareto optimal solution of MLP non-IID obtained by the FNSGA-III algorithm, two of which are solutions with a small global test error rate, and the other two are inflection point solutions. These 4 solutions are trained by federated learning and compared with the standard FedAvg algorithm, and the communication rounds are set to 150 rounds. In addition to increasing the number of communication rounds, each solution is verified under IID and non-IID, and it is investigated whether the non-dominated solutions obtained on the IID dataset are still valid on the non-IID dataset, and vice versa. All validation results are listed in Table 4.

Table 4 Experimental data of MLP non-IID solution obtained by FNSGA-III algorithm

parameter Solution 1 Solution 2 Solution 3 Solution 4 Standard FedAvg Participation ratio C 0.62 0.473 3 0.281 2 0.753 4 1 Learning rate η 0.112 5 0.114 4 0.085 0.094 9 0.05 SET parameter ε 95 31 31 3 / The number of neurons in the hidden layer of MLP 173 136 197 208 [200,200] communication cost 57 553 14 330 9 163 3 972 199 210 Test accuracy/% under non-IID 96.70 96.08 94.81 91.60 95.69 Test accuracy distribution variance under non-IID 4.71 5.79 12.78 22.50 9.86 Test accuracy/% under IID 97.98 97.58 97.45 95.39 97.35 Test accuracy distribution variance under IID 1.67 2.12 2.18 3.94 2.24

From the results shown in Table 4, the following observations can be made on the evolution of the four Pareto solutions of the selected MLP non-IID. In the experimental results of the four solutions obtained under MLP non-IID data distribution under the non-IID data distribution, the sparsity parameter of solution 4 is 3. Although its accuracy curve iteration is stable, it is obviously lower than other cases, which may mean excessive The ground sparse neural network will damage the model accuracy; Solution 1 and Solution 2 are better than the results of standard federated learning in terms of communication cost, accuracy and variance, and the iterative curve is stable, which verifies the Pareto solution obtained by the algorithm FNSGA-III proposed in the present invention. , there are high-quality solutions. Among the experimental results under the IID data distribution, only the iteration curve of solution 4 is inferior to standard federated learning. It can be said that the solution under MLP non-IID is valid under non-IID, and it can still have better running effect when extended to IID.

The beneficial effects of the present invention are as follows:

The present invention proposes the FNSGA-III algorithm to solve the multi-objective federated learning model problem, and conducts experimental verification under unstable communication conditions. We first constructed a three-objective template for federated learning, and set the optimization objective to minimize the global model test error rate, communication cost, and global model accuracy distribution variance. The decision variables were the hyperparameters of the neural network and the federated learning parameters. The NSGA-III algorithm is introduced to solve the federated learning multi-objective model, and the initialization of NSGA-III is changed. The experimental results show that the improved FNSGA-III algorithm is better than the original NSGA-III algorithm. And the Pareto optimal solution optimized by FNSGA-III algorithm is compared with the benchmark federated average algorithm, which effectively improves the accuracy of the global model, reduces the distribution variance of the accuracy of the global model and the communication cost.

As used herein, the word "preferred" means serving as an example, instance or illustration. Any aspect or design of the present invention described as "preferred" is not necessarily to be construed as advantageous over other aspects or designs. Rather, use of the word "preferred" is intended to introduce concepts in a specific manner. The term "or" as used in this application is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless specified otherwise or clear from context, "X employs A or B" is meant to naturally include either of the permutations. That is, "X uses A or B" is satisfied in any of the preceding examples if X uses A; X uses B; or X uses both A and B.

Furthermore, although the disclosure has been shown and described with respect to one implementation or implementation, equivalent variations and modifications will occur to those skilled in the art based on a reading and understanding of this specification and the accompanying drawings. The present disclosure includes all such modifications and variations and is limited only by the scope of the appended claims. In particular with respect to the various functions performed by the above-described components (eg, elements, etc.), the terms used to describe such components are intended to correspond to any component that performs the specified function of the component (eg, which is functionally equivalent) (unless otherwise indicated), even if not structurally equivalent to the disclosed structures that perform the functions of the illustrated exemplary implementations of the present disclosure by this disclosure. Furthermore, although a particular feature of the present disclosure has been disclosed with respect to only one of several implementations, such feature may be combined with one or other features of other implementations as may be desired and advantageous for a given or particular application combination. Also, to the extent that the terms "comprising", "having", "containing" or variations thereof are used in the detailed description or the claims, such terms are intended to be inclusive in a manner similar to the term "comprising".

Each functional unit in this embodiment of the present invention may be integrated into one processing module, or each unit may exist physically alone, or multiple or more units may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software function modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium. The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, and the like. The above-mentioned devices or systems can execute the storage methods in the corresponding method embodiments.

To sum up, the above-mentioned embodiment is an embodiment of the present invention, but the embodiment of the present invention is not limited by the embodiment, and any other changes that deviate from the spirit and principle of the present invention, Modifications, substitutions, combinations, and simplifications should all be equivalent substitutions, which are all included within the protection scope of the present invention.

Claims (10)

1. The multi-target federated learning evolution method based on the improved NSGA-III is applied to a server and a plurality of participants, and is characterized by comprising the following steps:

acquiring learning data, wherein the learning data is used for labeling;

constructing a multi-objective optimization model for federal learning, wherein the multi-objective optimization model comprises three objectives of maximizing the accuracy of a global model, minimizing the distribution variance of the accuracy of the global model and communication cost;

fast greedy initialization P _t Multi-target evaluation was performed using the FedAvg algorithm, 0;

to P _t Performing non-dominant sequencing;

performing iteration, wherein each iteration performs the following operations: selection, crossover, mutation operator operations, producing Q _t Carrying out federal learning training evaluation on the offspring population, and calculating to obtain three targets of each individual; mixed parent and child populations R _t ＝Q _t +P _t (ii) a To R _t Non-dominant ordering and selection of P by reference point _t+1 ；

Finding out the Pareto optimal solution, and outputting a labeling result corresponding to the Pareto optimal solution.

2. The multi-objective federated learning evolution method based on modified NSGA-III as claimed in claim 1Characterized in that, in the course of the federal learning evolution, the data set is owned as D _k The loss function for the kth participant of (1) is:

the global goal of the federated learning evolution approach is to minimize the global loss function l (w) as follows:

where k is the participant's serial number, L _k (w) is the loss function of the kth participant,/ _i (w) is a loss function on data sample i, n _k Data set D for participant k _k Size n _k ＝|D _k And n is the total size of the data samples of the K participants.

3. The multi-objective NSGA-III based federated learning evolution method of claim 1, wherein each participant receives the global model w from the server during each round of the federated learning process _t And training the global model by using the local data to obtain an updated local model

Then the participant sends the updated local model to the server, and the server aggregates the models according to a certain rule to obtain a new global model w _t+1 For the next round of iterative training, the subscript t represents the federally learned communication round.

4. The multi-objective federated learning evolution method based on modified NSGA-III as claimed in claim 1, wherein the three-objective optimization model of federated learning evolution is:

wherein F (v) is the objective function of a three-objective optimization model having 3 minimization objectives, each of which is the minimization of the global model test error rate f ₁ Global model accuracy distribution variance f ₂ Communication cost f ₃ Conv is the number of convolution layers, kc is the number of convolution kernels, ks is the size of convolution kernels, L is the number of fully-connected layers, N is the number of fully-connected layer neurons, eta is the learning rate, epsilon is the connectivity parameter of the neural network, the number of connections between two fully-connected layers is determined by the connectivity parameter epsilon, and the total number of connections is N ═ epsilon (N ═ epsilon) ^k +n ^{k -1} ) Wherein n is ^k And n ^k-1 The numbers of neurons in the k-layer and the k-1-layer, respectively.

5. The multi-objective NSGA-III based federated learning evolution method of claim 1, wherein objective f is ₁ The global model test error rate E is 1-A, A is the average test accuracy rate of the global model

{a ₁ ,a ₂ ,...,a _K The accuracy of each participant.

6. The multi-objective NSGA-III based federated learning evolution method of claim 5, wherein objective f is ₂ For global model accuracy distribution variance

7. The multi-objective NSGA-III based federated learning evolution method of claim 5, wherein objective f is ₃ Can be expressed as

K is the total number of participants, C is the participation proportion of each round of participants, and sigma is the size of the model parameter.

8. The multi-target federal learned evolution method based on improved NSGA-III as claimed in claim 1, wherein the process of performing federal learned training evaluation on the offspring population is implemented by a static SET-based FedAvg algorithm, which specifically includes the following steps:

i is an individual of a population in the FNSGA-III algorithm, P is the population scale, and after the individual i is decoded, the related federal learning neural network hyper-parameter, the connectivity of the neural network and the participation ratio C of each round of participants are obtained _i ；

Using a link communication parameter epsilon _i Initializing a static SET topology, and taking the static SET topology as a global model used in an algorithm;

in each round of training process, training local data by using a small batch random gradient descent method;

after a certain turn, three targets of the test error rate of the global model, the accuracy rate distribution variance of the global model and the communication cost are calculated.

9. The multi-target federal learning evolution method based on modified NSGA-III as claimed in claim 1, wherein the number of convolution layers, the number of convolution kernels, the size of convolution kernels, the number of full-link layers, the number of neurons in each layer of full-link layers and the SET parameter epsilon of MLP and CNN are encoded using binary, and the learning rate η and the participation ratio C in each round are encoded using real values.

10. The multi-objective NSGA-III based federated learning evolution method of claim 1, wherein the learning data is MNIST data set.

Images