CN117221122B - Asynchronous layered joint learning training method based on bandwidth pre-allocation - Google Patents

Abstract

The invention discloses an asynchronous hierarchical joint learning training method based on bandwidth pre-allocation, which comprises the following steps: when training is started, the cloud server selects a corresponding number of clients in the range of each edge server, and distributes the latest model parameters to the clients; the client performs multiple iterations on the model parameters by utilizing own data, selects the nearest edge server with the residual bandwidth according to the current position of the client to upload the model parameters when training is finished, and reasonably distributes resources according to the bandwidth condition to accelerate the uploading speed; each edge server at the lapse of a certain time interval T _aggregation Then carrying out one round of edge aggregation, and uploading the aggregated parameters to a cloud server for one round of cloud aggregation; after cloud aggregation, the cloud server performs selection of the next round of clients. The invention not only adapts to the change of data distribution among the participants in a dynamic scene, but also fully utilizes limited communication resources, thereby improving the training effect.

Description

An asynchronous hierarchical joint learning training method based on bandwidth pre-allocation

Technical field

The invention relates to an asynchronous hierarchical federated learning training method based on bandwidth pre-allocation, and belongs to the technical field of federated learning model training.

Background technique

The emergence of digital technology has driven significant progress in various transformative technologies such as big data and artificial intelligence. Machine learning-powered mobile apps are revolutionizing every aspect of modern life. However, machine learning training tasks often require large amounts of data from various terminals with different computing capabilities. The traditional method is to upload data to a remote cloud server for processing, but this method has challenges such as privacy violations, network congestion, and transmission delays, which hinder the full utilization of the data.

In 2016, Google Research proposed the concept of federated learning, aiming to alleviate network bandwidth constraints and solve data privacy vulnerabilities. Federated learning is a collaborative training and sharing method that eliminates the need to access raw data, conforms to the principles of decentralized collection and data minimization, and updates are shared with a central server or coordinator. This decentralized approach follows data minimization principles and eliminates the need for centralized data collection. The raw data remains securely stored on the individual device and cannot be accessed directly. Federated learning enables local model training on each device while only uploading aggregated updates or model parameters to a central server.

Hierarchical federated learning is a federated learning framework that typically deploys edge servers on base stations. In this framework, edge servers are deployed on base stations and serve as intermediate stations between mobile devices and cloud servers. These edge servers facilitate the aggregation of local models of neighboring devices received from the edge. Hierarchical federated learning successfully solves the challenge of uploading data to the cloud by enabling cloud servers to efficiently process data from more terminals.

However, standard hierarchical federated learning frameworks employ synchronous aggregation of global models, where the server waits for all client parameters to be uploaded before performing global aggregation. This synchronization method leads to a "straggler effect", where the latency of global aggregation is determined by the client with the slowest upload parameters, resulting in increased latency throughout the training process. Furthermore, delayed aggregation of client-side parameters hinders the convergence of the global model and may affect the accuracy and performance of the trained model.

Therefore, it is necessary to propose an asynchronous hierarchical federated learning framework that eliminates the need for the server to wait for all client parameters to be uploaded before global aggregation, thereby shortening the training delay of federated learning.

Contents of the invention

In order to solve the above existing problems, the present invention discloses an asynchronous hierarchical joint learning training method based on bandwidth pre-allocation. The specific technical solution is as follows:

An asynchronous hierarchical joint learning training method based on bandwidth pre-allocation, including the following steps:

Step 1: Cloud server selects client

In this step, the cloud server is responsible for coordinating and managing joint learning tasks. The cloud server selects a corresponding number of clients from each edge server according to the policy. If the number of optional clients is more than the required clients, the cloud server selects a corresponding number of clients based on the client Based on the quality and computing power of local training data of the client, the best clients are selected to participate in the next round of training and the model parameters are sent to these clients;

Step 2: Client local training

After receiving the model parameters, the selected client will perform model training locally. Each client executes an optimization algorithm according to the task type and model architecture to update the parameters of its local model. This process can carry out multiple rounds of iterations, each iteration The model parameters will be updated;

Step 3: Upload local parameters

After the client iterates the local model to the preset accuracy, the client selects the nearest edge server with remaining bandwidth to associate based on the current geographical location, and then uploads the local model parameters to the edge server;

Step 4: Edge aggregation

Model parameter aggregation is a federated learning method that generates a better global model by averaging, weighted averaging or other aggregation strategies on parameters from different clients; unlike other hierarchical federated learning frameworks, edge servers do not require Collect all model parameters sent by the associated clients and then aggregate them. Instead, aggregate the collected model parameters after a period of Tag _aggregation ; considering that the collected model parameters may be outdated data generated by the previous round of training , so the weight will be given based on the time when the parameters start to iterate locally, and the earlier the parameter, the smaller the weight; each edge server will send the aggregated model parameters to the cloud server after edge aggregation;

Step 5: Cloud Aggregation

Every time T _aggregation elapses, the cloud server receives the model parameters sent by all edge servers at the same time, and aggregates these model parameters. If the aggregation model reaches the specified accuracy, the asynchronous hierarchical joint learning training based on bandwidth pre-allocation is stopped. Otherwise return to step 1.

Furthermore, in step 1, if the client moves in a wide range during the training process, the movement of the clients selected by the cloud server in each round during the training process cannot be predicted in advance. When the training ends, these clients will Will be associated to different edge servers; due to the limited bandwidth of each edge server, too many associated clients will cause some clients to be unable to upload model parameters smoothly, and fewer clients will lead to poor quality of edge model aggregation, so before training When selecting clients, you need to select a corresponding number of clients within each edge server;

The specific method of client selection is:

Historical statistical patterns can be found in the movement of clients during the training process. For example, a client from a residential area under the scope of edge server A will stay there during the training process, go to a business area under the scope of edge server B, or go to a client under the scope of edge server C. The corresponding probability distribution can be found for the company's work situation. In this case, the transfer probability of each edge server to another edge server during the training process can be represented by a matrix. The actual investigation obtains the client transfer within the coverage of the cloud server. The matrix of the situation, based on this matrix and the number of clients that need to be associated with each edge server, calculate the number of clients that the cloud server initially selects under each edge server, so that the number of clients associated with each edge server is even when the training ends;

As shown in the matrix, the elements in the jth row and column l represent the probability that the client transfers from the edge server j range to the edge server l range during the training process.

The initial bandwidth allocation ratio of each client participating in training is β _device , and the cloud server is based on the remaining bandwidth allocation ratio of each edge server j Related/> Clients are uploaded to the server at the end of training. Assume that the number of clients that |K| edge servers should be associated with in the next round is (n ₁ ,n ₂ ,n ₃ ,...,n _|K| ), The number of clients X=(x ₁ ,x ₂ ,x ₃ ,…,x _|K| ) that the cloud server initially allocates under |K| edge servers makes xM=(y ₁ ,y ₂ ,y ₃ ,.. .,y _|K| ), the following optimization problem is obtained:

St for/> is the number of remaining clients under server j,

As a general constraint problem, it is solved by the multiplier method-PHR algorithm. Given the initial point (n ₁ ,n ₂ ,n ₃ ,...,n _|K| ), the penalty factor σ, the amplification coefficient c ₁ >1, Control error ε>0, constant θ∈(0,1), let k=1, the solution process is as follows:

Step 1: Taking x _k-1 as the initial point, solve the unconstrained problem:

Obtain the optimal solution x _k ;

Step2:If Then x _k is the optimal solution and stop; otherwise, go to Step 3;

Step3: If Go to Step 4; otherwise let σ _k+1 = cσ _k and go to Step 4;

Step4: Correct the multiplier vector:

(λ _k+1 ) ₁ = (λ _k ) ₁ -σc ₁ (x _k )

(λ _k+1 ) _i =max[0,(λ _k ) _i -σc _i (x _k )],i＝2,3,...,|2K+1|,

Let k=k+1, go to Step 2;

c _i (x) is the i-th constraint, the 1st is an equality constraint, the 2nd to 2|K|+1 are inequality constraints, iterate according to this algorithm and obtain the approximate optimal solution, The precision ε does not need to be very high, and the calculated X is rounded to an integer; the optimal solution is the optimal number of clients that the cloud server needs to select in each edge server range.

Further, in step 2, the client participating in the training receives the model parameters from the cloud server, and the client i uses its own data set D _i to solve for the optimal model parameters ω to represent so that the loss function Minimum, then the loss function on client i is expressed as: The goal is to find the optimal model parameters ω ^* =argmin[F _i (ω)] that minimize the loss function;

This problem is almost unsolvable. The client needs to perform gradient descent in multiple iterations to gradually approach the optimal solution. In order to achieve the predetermined local accuracy θ∈(0,1), the client needs to perform multiple rounds of local iterations L(θ) =μlog(1/θ), where the constant μ depends on the size of the training task. The nth _{LocalTraining} round of local iteration is expressed as: Iterate until/> When, local training is completed, where eta is the learning rate;

The computing delay of model training on the client is related to the number of iterations L(θ) required to achieve accuracy, the computing power _Pi of the client and the size of the training data set |D _i |, expressed as: The computing power P is the number of samples processed by the client within a period of time. The size of the training data set can be set in advance, and the calculation delay can be effectively reduced by selecting a client with strong computing power.

Further, in step 3, after the client performs local training and reaches the predetermined accuracy, it immediately uploads the model parameters to the edge server. Under the framework of asynchronous hierarchical federated learning based on bandwidth pre-allocation, client i is not initially associated with the corresponding edge server, but at the end of training, the distance s _ij to each edge server j is calculated based on the current location, so that there is still remaining bandwidth, that is Select the edge server j with the closest distance, that is, mins _ij , to associate among the servers, and upload the model parameters to the edge server.

The initial bandwidth allocation ratio of the associated edge server to all clients is β _ji , then the parameter upload rate of client i is: where h _i is the channel gain of client i, N ₀ is the noise,/> is the receiving power of edge server j to client i, expressed as/> Where p _j is the maximum received power of edge server j, and c is a constant, then the client parameter upload delay/> Where |d _i | is the size of the model parameters that need to be uploaded.

Furthermore, in step 4, in the synchronous aggregation algorithm, the cloud server needs to receive the parameters of all clients participating in the training before performing a round of global aggregation, resulting in a global aggregation delay caused by the last one completing training and uploading parameters. The client makes the decision, resulting in a "straggler effect". The client's local data set size and computing performance vary greatly. The client selected by the initial cloud server is not associated with the edge server. Therefore, the edge server's synchronization and aggregation method may cause a delay in waiting for the corresponding response. The client upload seriously affects the model training progress, so the edge asynchronous aggregation method is adopted. Each edge server aggregates the collected model parameters after a period of Tag _aggregation and uploads the aggregated parameters to the cloud server. .

Furthermore, when the asynchronous aggregation method is used, the model parameters uploaded by the client may be out of date. Therefore, the stale function of each client when performing edge aggregation is different from the number n _round of cloud aggregation rounds that have been performed when the client receives the model. It is related to the latest round number n _CurrentRound of cloud aggregation, so the stale function corresponding to the parameters uploaded by client i is expressed as Where λ∈(0,1) is the given attenuation coefficient, and the parameter update on edge server j is expressed as: Where S _j represents the set of clients participating in this edge aggregation.

Furthermore, in step 5, considering that the edge server and the cloud server have strong communication capabilities, the communication delay for the edge server to upload parameters to the cloud server is negligible. After the cloud server receives the parameters uploaded by the edge server, Immediately perform a round of cloud aggregation;

Also because the quality of parameters received by cloud aggregation is uneven, it is necessary to introduce the stale function as a hyperparameter to reduce the impact of outdated models on global model training. The stale function of parameters uploaded by each edge server is consistent with the participating customers in the last round of aggregation of the edge server. It is related to the overall staleness of the terminal, which can be simply set as: The update of model parameters on the cloud server is expressed as:/> Among them/> Represented as the latest model parameters uploaded by edge server j, D _s is the total set of data sets of clients that have participated in training. These model parameters are aggregated. If the aggregation model reaches the specified accuracy, the asynchronous analysis based on bandwidth pre-allocation is stopped. Layer joint learning training, otherwise return to step 1.

Furthermore, when the communication capabilities of the edge server and the cloud server are relatively strong, the edge server does not perform edge aggregation after receiving the parameters uploaded by the client, but directly sends these parameters to the cloud server for cloud aggregation. In this way, the staleness function The impact of model parameter weights is more accurate.

Furthermore, when selecting a client, the cloud server needs to consider the computing performance of the client to reduce local computing latency; the data quality of the data set on the client can enable the model to achieve better local training effects because it is representative and diverse. Specific data can improve the model training effect. To this end, the following technical methods are adopted: when the number of clients that can participate in training is more than the clients required in a round of training, the computing performance of the client and the entropy weight of the data are comprehensively considered. Select the best client under each edge server to participate in the next round of training;

The data quality of the data set on the client is defined using the entropy weight method. m samples are extracted from client i, 1,2,…,k _i represents the attribute feature index on client i. 1,2,…,m is the sample index,/> Is the normalized value of the data attribute,/> The entropy weight assigned to the data attribute, is the information entropy of each data attribute;

Under each server j, select A client with optimal computing power P and local model quality Q. Assume that P and Q change little in multiple rounds of training under the same client. The overall strategy is to give priority to each edge server from the client set N that has participated in training. Select clients with larger comprehensive capabilities φ = γP + (1-γ) Q value from _selected , and the remaining required clients are selected from the total set of training clients N _sum , and are selected from the range of edge server j/> For a client and parameter δ, the specific steps are as follows:

S1: Update |N _selected |. If |N _selected | is greater than the set threshold θ _client , update the δ |N _selected | clients with the largest φ value in |N _selected | to join N _best , and then select all clients in N _best Clients in and within the range of edge server j indivual;

S2: If Then it is in N _sum -N _selected , and meets the condition of s _ji <R _j , that is, it is randomly selected within the range of edge server j/> clients, calculate the φ values of these clients and add them to N _selected . All clients in N _selected are the clients participating in the next round of training.

Furthermore, the cloud server selects the number of clients after an edge server performs edge aggregation and uploads parameters. This edge server definitely has remaining bandwidth. Other edge servers may also have remaining bandwidth, because some of them have strong computing capabilities and are not related to the edge servers. Clients that are closer to the server have completed local training, uploaded parameters, and released bandwidth. This bandwidth will also be allocated by the cloud server to the clients participating in the next round of training;

After the cloud server selects a client, the remaining bandwidth ratio of all edge servers is set to 0. Considering that some clients cannot complete training and release bandwidth for a long time due to abnormal reasons, the model parameters of these clients will also be outdated after completing training, so in After the set n _overtime round of cloud aggregation, the allocated but not released bandwidth will be automatically released for the next round of client parameter uploads.

The beneficial effects of the present invention are:

Compared with the existing technology, the beneficial effects of the present invention are: after the client training is completed, the present invention associates it with the nearest edge server with remaining bandwidth according to the current location to shorten the model upload delay; in order to avoid uneven distribution of clients Before each round of training, the number of clients initially assigned to each edge server in this round is calculated based on the remaining bandwidth of each edge server and the client transfer matrix; when selecting clients, the pairs are selected based on the entropy weight calculation. Model training is helpful and the client with strong computing power is used to shorten the local training delay; during the upload process, the upload is accelerated according to the remaining bandwidth of the edge server to further shorten the upload delay.

Description of drawings

Figure 1 is a training flow chart of the present invention,

Figure 2 is a cloud interaction diagram of the present invention,

Figure 3 is a diagram showing the performance of the model accuracy in the test set in the comparative experiment in the embodiment.

Figure 4 is a diagram showing the performance of the model loss rate in the test set in the comparative experiment in the embodiment.

Figure 5 is a comparison of the time required for users to upload data in the greedy algorithm and the hierarchical federated learning algorithm in the embodiment.

Detailed ways

The present invention will be further elucidated below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention.

It can be seen with reference to the accompanying drawings 1-2 that the process of the present invention is:

1) Cloud server selects client

Select a corresponding number of clients under each edge server, distribute the latest model parameters to these clients, and use local data sets for local training.

2)Client local training

In this step, the client participating in the training receives the model parameters from the cloud server, and the client uses the local data set to solve the optimal model parameters.

3)Local parameter upload

After the client performs local training and reaches the predetermined accuracy, it immediately uploads the model parameters to the edge server.

4) Edge aggregation

In the synchronous aggregation algorithm, the server needs to receive the parameters of all clients participating in the training before performing a round of global aggregation. As a result, the delay of global aggregation is determined by the last client that completes training and uploads parameters, resulting in a "straggler effect" . In the scenario of the present invention, the client's local data set size and computing performance vary greatly. The client selected by the initial cloud server is not associated with the edge server. Therefore, the synchronous aggregation method of the edge server may cause a delay in waiting for the corresponding client to upload, which seriously affects model training progress. Therefore, the present invention is different from other hierarchical federated learning frameworks. The edge server does not need to collect all the model parameters sent by the associated clients and then aggregate them. Instead, it aggregates the collected model parameters after a period of _taggration ; Considering that the collected model parameters may be outdated data generated by the previous round of training, weights will be given based on the time when the parameters start to iterate locally. The earlier the parameters, the smaller the weight; after each edge server is aggregated at the edge, it will be aggregated The final model parameters are sent to the cloud server;

When using asynchronous aggregation, the model parameters uploaded by the client may be out of date. Therefore, the stale function of each client when performing edge aggregation is different from the number n _round of cloud aggregation that has been performed when the client receives the model and the latest cloud aggregation. The number of rounds n is related to _CurrentRound , so the stale function corresponding to the parameters uploaded by client i is expressed as Where λ∈(0,1) is the given attenuation coefficient, and the parameter update on edge server j is expressed as: Where S _j represents the set of clients participating in this edge aggregation.

5) Cloud aggregation

Considering that the edge server and the cloud server have strong communication capabilities, the communication delay for the edge server to upload parameters to the cloud server is negligible. After the cloud server receives the parameters uploaded by the edge server, it immediately performs a round of cloud aggregation. Also because the quality of parameters received by cloud aggregation varies, the staleness function needs to be introduced as a hyperparameter to reduce the impact of outdated models on global model training. The staleness function of each edge server's upload parameters is related to the overall staleness of the participating clients in the last round of aggregation of the edge server, and can be simply set as: The update of model parameters on the cloud server is expressed as:/> Among them/> Represented as the latest model parameters uploaded by edge server j, D _s is the total set of data sets of clients that have participated in training. These model parameters are aggregated. If the aggregation model reaches the specified accuracy, the asynchronous analysis based on bandwidth pre-allocation is stopped. Layer joint learning training, otherwise the cloud server will select the next round of clients and distribute the latest parameters to these clients.

If the communication capabilities of the edge server and the cloud server are relatively strong, the edge server can not perform edge aggregation after receiving the parameters uploaded by the client, but directly send these parameters to the cloud server for cloud aggregation. In this way, the staleness function will be useful for the model. The influence of parameter weights is more accurate.

In addition, after the cloud server selects the number of clients, after an edge server performs edge aggregation and uploads parameters, this edge server definitely has remaining bandwidth. Other edge servers may also have remaining bandwidth, because some of them have strong computing power and are different from the edge servers. Clients that are closer have completed local training, uploaded parameters, and released bandwidth. This bandwidth will also be allocated by the cloud server to the clients participating in the next round of training. After the cloud server selects the client, the remaining bandwidth ratio of all edge servers is set to 0. Considering that some clients cannot complete training and release bandwidth for a long time due to abnormal reasons, the model parameters of these clients are also outdated after completing training, so within a period of time The bandwidth allocated after the time and not released will be automatically released for the next round of client parameter uploads.

In order to verify the application of this patent, the specific experiments of this patent are given below:

lab environment:

The experiment considers training under a distributed framework consisting of a cloud server, 5 edge servers and 250 clients to participate in training. Each edge server contains 50 clients. In edge-level federated learning, 10 clients under each edge will be randomly selected to participate in training. In cloud-level federated learning, the parameters of each edge server are involved. Cloud aggregation.

In local training, the Lenet convolutional neural network was considered as a model and verified on the minst data set. The model is initialized with two convolutional layers, one dropout layer and two fully connected layers. The number of input channels of the first convolutional layer is 3, the number of output channels is 10, and the convolution kernel size is 5. The number of input channels of the second convolutional layer is 10, the number of output channels is 20, and the convolution kernel size is 5. The number of input nodes of the first fully connected layer is 320, and the number of output nodes is 50. The number of input nodes of the second fully connected layer is 50, and the number of output nodes is 10. The input data is passed through the first convolutional layer, followed by max pooling and ReLU activation. Then, the data passes through the second convolution layer and dropout layer, and then undergoes max pooling and ReLU activation. The data is then flattened and passed through two fully connected layers, and the output is returned.

This example randomly selects 1,000 images from the 60,000 labeled data set and distributes them to the cloud. The 5,000 images are evenly divided into 5 parts and distributed to 5 edge servers for testing the accuracy of the model at the edge and the cloud. The remaining 54,000 images are evenly distributed to 250 clients. In this embodiment, the independent and identically distributed method is used for allocation. The edge server and the client allocate uniformly and randomly during the process of allocating samples.

This embodiment simulates the time it takes for the model parameters to be uploaded to the edge server after local training is completed. The distance from all clients to the edge server is standardized to be between (0,1). Considering that the signal strength is only related to the weakening of the free path, assuming that the signal-to-noise ratio of the farthest client within the range of the edge server is 5, then according to the maximum information in the channel Transmission rate formula, upload delay can be expressed as

In the experiment, the learning rate of the mnist data set was 0.01. During the experiment, the learning rate in each round was attenuated to 0.995 times the previous one. During each round of local training, the model was iterated locally for 40 rounds.

Compare experimental settings:

·Hierarchical federated learning (control): Before the start of training and after each round of cloud aggregation, 5 edge servers randomly select 10 clients within the range to participate in training. The clients may move during the training process. When completing local training, if Within the scope of the associated edge server, the model parameters are uploaded to the corresponding edge server. When the edge server collects all clients or exceeds the maximum waiting time, it performs a round of edge aggregation and uploads the aggregated model parameters to the cloud server. The cloud server collects the parameters from all edge servers and performs a round of cloud aggregation.

·When uploading model parameters, the client associates with the nearest server (control): Before the start of training and after each round of cloud aggregation, 10 clients are randomly selected for training within the range of each edge server. The client may move during the training process. After the local training is completed, it selects the edge server closest to it to associate with it and upload the model parameters. After a certain period of time, each edge server performs edge aggregation on the model parameters that have been uploaded. Parameters that failed to be uploaded in time were put into the buffer during the experiment. These parameters will participate in the subsequent edge aggregation based on the upload delay. Buffering The parameters in the zone are corrected each round based on the stale function. The cloud server can receive the model parameters and the number of participating clients uploaded by all edge servers almost simultaneously for cloud aggregation.

·Asynchronous hierarchical federated learning based on bandwidth pre-allocation: Before the start of training and after each round of cloud aggregation, the number of clients selected within each edge server is calculated based on the remaining bandwidth and transfer matrix of each edge server, and The remaining bandwidth of all edge servers is set to 0. The client may move during the training process. After the local training is completed, the upload delay is calculated based on the remaining bandwidth and distance within each edge server where it is located, and the edge server with the smallest delay is selected to associate and upload the model parameters. (bandwidth

=min (customize the maximum allocated bandwidth, remaining bandwidth + 1), after the upload is completed, the remaining bandwidth of the edge server + 1. If the client fails to upload to the pre-allocated edge server within a period of time, the edge server automatically releases the bandwidth. After a certain period of time, each edge server performs edge aggregation on the uploaded model parameters and uploads them to the cloud server. The cloud server receives the model parameters and the number of participating clients uploaded by all edge servers and performs cloud aggregation.

Conclusion analysis:

The experimental results are shown in Figures 3 and 4. Since the time of each round of aggregation in hierarchical federated learning is not fixed, the experiment uses the training time as the abscissa and the time of each round of cloud aggregation in the algorithm of this embodiment as a time unit. The vertical coordinates are the accuracy and loss values of the model after cloud aggregation on the test set at the corresponding time.

It can be seen from the figure that although the algorithm of this embodiment uses an asynchronous aggregation algorithm, it is not as good as the synchronous algorithm in terms of model convergence and stability. However, in this scenario, the model reaches an accuracy of 0.9. Hierarchical federated learning requires 113 time units. The algorithm in this embodiment only requires 33 time units, can converge quickly, and reaches an accuracy of 0.95 in 108 time units. , achieve better results. The experiment of this embodiment does not consider the delay of local iteration, and the actual effect will not be so obvious.

In this scenario of the hierarchical federated learning algorithm, due to the movement of clients during the training process, the model parameters may not be uploaded to the corresponding edge server on time. When the maximum waiting time is set, some clients may not participate in the end. The aggregation of edge models leads to instability of the curve and poor final aggregation effect.

When uploading model parameters, the client associates with the nearest server, which can greatly shorten the delay in uploading model parameters in the framework of hierarchical federated learning. As shown in Figure 5, the abscissa in the figure is the time required for the client to upload the model to the associated edge server. Time, the ordinate is the time required to upload to the nearest edge server, and the maximum value is normalized to 1. However, this method may cause the initially allocated clients to gather within the range of some edge servers after training is completed, causing these edge servers to need to receive additional clients. If bandwidth cannot be allocated, the clients will be blocked in the upload stage. Finally, It affects the convergence effect of the model and causes the curve to be unstable.

Asynchronous hierarchical federated learning based on bandwidth pre-allocation processes the allocation of initial clients based on the greedy algorithm, so that when local training is completed, the number of clients under each edge server is relatively even, making the model convergence process more stable. After collecting parameters from clients with short upload delays, the edge server will allocate more bandwidth to clients with longer upload delays, speeding up the upload of model parameters, so that the model parameters of more clients can participate in edge aggregation in a timely manner.

The technical means disclosed in the solution of the present invention are not limited to the technical means disclosed in the above technical means, but also include technical solutions composed of any combination of the above technical features.

Taking the above-mentioned ideal embodiments of the present invention as inspiration and through the above description, relevant workers can make various changes and modifications without departing from the scope of the technical idea of the present invention. The technical scope of the present invention is not limited to the content in the description, and must be determined based on the scope of the claims.

Claims (7)

1. An asynchronous hierarchical joint learning training method based on bandwidth pre-allocation, which is characterized by including the following steps: Step 1: Cloud server selects client In this step, the cloud server is responsible for coordinating and managing the joint learning task. The cloud server selects a corresponding number of clients from the range of each edge server according to the policy. Based on the situation that the number of optional clients exceeds the required clients, the cloud server selects a corresponding number of clients based on the policy. Select the client to participate in the next round of training based on the quality and computing power of the client's local training data, and send the model parameters to the client to participate in the next round of training. Based on the client's computing performance and data entropy weight, in Select clients from the edge server to participate in the next round of training; The specific method of client selection is: During the training process, the transfer probability of each edge server to another edge server can be represented by a matrix. Actual investigation obtains the matrix of client transfer situations within the coverage of the cloud server. Based on this matrix and the number of clients that need to be associated with each edge server, calculate The Izumo server initially selects the number of clients under each edge server so that the number of clients associated with each edge server is even at the end of training; As shown in the matrix, the element e in the jth row and column l represents the probability that the client transfers from the edge server j range to the edge server l range during the training process. The initial bandwidth allocation ratio of each client participating in training is β _device , and the cloud server is based on the remaining bandwidth allocation ratio of edge server j Related/> Clients are uploaded to the server at the end of training. Assume that the number of clients that |K| edge servers should be associated with in the next round is (n ₁ ,n ₂ ,n ₃ ,...,n _K ). The cloud server _The number of clients initially allocated under |K| edge servers X=(x _{1 ,} x ₂ ,x ₃ ,..., _{x |K|} ), so that .,y _|K| ), the following optimization problem is obtained: for/> is the number of remaining clients under server j, As a general constraint problem, it is solved by the multiplier method-PHR algorithm. Given the initial point (n ₁ ,n ₂ ,n ₃ ,...,n _|K| ), the penalty factor σ, the amplification coefficient c ₁ >1, Control error ε>0, constant θ∈(0,1), let k=1, the solution process is as follows: Step 1: Taking x _k-1 as the initial point, solve the unconstrained problem: Obtain the optimal solution x _k ; Step2:If Then x _k is the optimal solution and stop; otherwise, go to Step3; Step3: If Go to Step 4; otherwise let σ _k+1 = cσ _k and go to Step 4; Step4: Correct the multiplier vector: (λ _k+1 ) ₁ = (λ _k ) ₁ -σc ₁ (x _k ) (λ _k+1 ) _i =max[0,(λ _k ) _i -σc _i (x _k )],i=2,3,...,2|K|+1, Let k=k+1, go to Step 2; c _i (x) is the i-th constraint, the 1st is an equality constraint, the 2nd to 2|K|+1 are inequality constraints, iterate according to this algorithm and obtain the approximate optimal solution, The accuracy ε does not need to be very high, and the calculated x is rounded to an integer; the optimal solution is the optimal number of clients that the cloud server needs to select in each edge server range; Step 2: Client local training After receiving the model parameters, the selected client will perform model training locally. Each client executes an optimization algorithm according to the task type and model architecture to update the parameters of its local model. This process can carry out multiple rounds of iterations, each Model parameters will be updated in every iteration; Step 3: Upload local parameters After the client iterates the local model to a preset accuracy, it associates the client with the edge server based on the distance between the client's geographical location and the edge server containing the remaining bandwidth; After the client performs local training to achieve a predetermined accuracy, it uploads the model parameters to the edge server. Under the framework of asynchronous hierarchical federated learning based on bandwidth pre-allocation, client i is not initially associated with the corresponding edge server, but is training At the end, the distance s _ij to each edge server j is calculated based on the current location, so that there is still remaining bandwidth, that is Select the edge server j with the closest distance, that is, mins _ij , to associate among the servers, and upload the model parameters to the edge server. The initial bandwidth allocation ratio of the associated edge server to all clients is β _ji , then the parameter upload rate of client i is: where h _i is the channel gain of client i, N ₀ is the noise,/> is the receiving power of edge server j to client i, expressed as/> Where p _j is the maximum received power of edge server j, and c is a constant, then the client parameter upload delay/> Where |d _i | is the size of the model parameters that need to be uploaded; Step 4: Edge aggregation The model parameter aggregation is a federated learning method that generates a better global model by averaging, weighted averaging or other aggregation strategies on parameters from different clients; different from other hierarchical federated learning frameworks, edge servers There is no need to collect all the model parameters sent by the associated clients and then aggregate them. Instead, T _aggregation will aggregate the collected model parameters after a period of time; based on the fact that the collected model parameters are outdated data generated by the previous round of training In the case of , weight is given to the local iteration time, and the earlier parameters have smaller weights; each edge server sends the aggregated model parameters to the cloud server after aggregation at the edge; Step 5: Cloud Aggregation Every time T _aggregation elapses, the cloud server simultaneously receives the model parameters sent by all edge servers, aggregates the parameter models sent by the edge servers, and stops asynchronous stratification based on bandwidth pre-allocation until the aggregated model reaches the predetermined accuracy. Joint learning training, otherwise return to step 1. 2. The asynchronous hierarchical joint learning training method based on bandwidth pre-allocation according to claim 1, characterized in that in step 2, the client participating in the training receives the model parameters transmitted from the cloud server, and the client i Use its own data set D _i to solve for the optimal model parameters ω to represent making the loss function/> Minimum, then the loss function on client i is expressed as:/> The client performs gradient descent in multiple iterations to gradually approach the optimal model parameters ω ^* =argmin[F _i (ω)] that minimize the loss function; In order to achieve the predetermined local accuracy θ∈(0,1), the client needs to perform multiple rounds of local iterations L(θ)=μlog(1/θ), where the constant μ depends on the size of the training task. The nth _{LocalTraining} round local The iteration is expressed as: Iterate until/> When, local training is completed, where eta is the learning rate; The computing delay of model training on the client is related to the number of iterations L(θ) required to achieve accuracy, the computing power of the client _Pi and the size of the training data set |D _i |. The computing delay of client i is expressed as : The computing power P is the number of samples processed by the client within a period of time. The size of the training data set is set in advance, and the calculation delay is effectively reduced by selecting a client with strong computing power. 3. The asynchronous hierarchical joint learning training method based on bandwidth pre-allocation according to claim 1, characterized in that "when using the asynchronous aggregation method, the staleness function is introduced as a hyperparameter to reduce the impact of the stale model on the global model. Impact, the cloud server distributes the latest model to the clients participating in the next round of training; when performing edge aggregation, the stale function of each client is related to the number of cloud aggregation rounds that have been performed when each client receives the model. n _round is related to the latest round number n _CurrentRound of cloud aggregation, so the stale function corresponding to the parameters uploaded by client i is expressed as Where λ∈(0,1) is the given attenuation coefficient, and the parameter update on edge server j is expressed as: Where S _j represents the set of clients participating in this edge aggregation. 4. The asynchronous hierarchical joint learning training method based on bandwidth pre-allocation according to claim 1, characterized in that in step 5, considering that the edge server and the cloud server have strong communication capabilities, the edge server The communication delay in uploading parameters to the cloud server is negligible. After the cloud server receives the parameters uploaded by the edge server, it performs a round of cloud aggregation; Also because the quality of parameters received by cloud aggregation is uneven, it is necessary to introduce the stale function as a hyperparameter to reduce the impact of outdated models on global model training. The stale function of parameters uploaded by each edge server is consistent with the participating customers in the last round of aggregation of the edge server. It is related to the overall staleness of the end, and the staleness function is set to: The update of model parameters on the cloud server is expressed as:/> Among them/> Represented as the latest model parameters uploaded by edge server j, D _s is the total collection of data sets of clients that have participated in training. These model parameters are aggregated until the asynchronous analysis based on bandwidth pre-allocation is stopped until the aggregated model reaches the predetermined accuracy. layer joint learning training, otherwise return to step 1. 5. The asynchronous hierarchical joint learning training method based on bandwidth pre-allocation according to claim 4, characterized in that, after receiving the parameters uploaded by the client, the edge server sends the parameters to the cloud server for cloud aggregation. 6. The asynchronous hierarchical joint learning training method based on bandwidth pre-allocation according to claim 1, characterized in that, "based on the computing performance and data entropy weight of the client, select the clients participating in the next round of training in the edge server. end; The data quality of the data set on the client is defined using the entropy weight method. m samples are extracted from client i, 1,2,…,k _i represents the attribute feature index on client i, 1,2,…,m is the sample index,/> Is the normalized value of the data attribute,/> The entropy weight assigned to the data attribute, is the information entropy of each data attribute; Under each server j, select based on computing power P and local model quality Q Clients, assuming that P and Q under the same client change little in multiple rounds of training, the overall strategy is for each edge server to _first select the comprehensive capability φ=γP+(1- γ) Clients with a large Q value, where γ is the weight coefficient, and the remaining required clients are selected from the total set of training clients N _sum , and are selected from the range of edge server j/> For a client and parameter δ, the specific steps are as follows: S1: Update |N _selected |. If |N _selected | is greater than the set threshold θ _client , update the δ |N _selected | clients with the largest φ value in |N _selected | to join N _best , and then select all clients in N _best Clients in and within the range of edge server j indivual; S2: If Then it is in N _sum -N _selected , and meets the condition of s _ji <R _j , that is, it is randomly selected within the range of edge server j/> clients, calculate the φ values of these clients and add them to N _selected . All clients in N _selected are the clients participating in the next round of training. 7. The asynchronous hierarchical joint learning training method based on bandwidth pre-allocation according to claim 6, characterized in that the cloud server selects the number of clients after one edge server performs edge aggregation and uploads parameters, and the other includes The remaining bandwidth of the edge server with remaining bandwidth will be allocated by the cloud server to the clients participating in the next round of training; After the cloud server selects the client, the remaining bandwidth ratio of all edge servers is set to 0. After the set t _overtime cloud aggregation, the allocated but not released bandwidth is automatically released, and the allocated but not released bandwidth is automatically released. The released bandwidth is used for the next round of client parameter uploads.