CN111401552B - Federal learning method and system based on batch size adjustment and gradient compression rate adjustment - Google Patents

Abstract

The invention discloses a federated learning method and system based on adjusting batch size and gradient compression rate, which are used to improve model training performance, including: in a federated learning scenario, multiple terminals share uplink wireless channel resources, and local terminal-based training Data, together with the edge server to complete the training of the neural network model; during the model training process, the terminal uses a batch method to calculate the gradient in the local calculation, and in the uplink transmission process, the gradient needs to be compressed before transmission; according to the calculation of each terminal Ability and its channel state, adjust the batch size and gradient compression rate to improve the convergence rate of model training while ensuring the training time and not reducing the accuracy of the model.

Description

Federated learning method and system based on adjusting batch size and gradient compression rate

Technical Field

The present invention relates to the field of artificial intelligence and communications, and in particular to a federated learning method and system based on adjusting batch size and gradient compression rate.

Background Art

In recent years, with the continuous improvement of hardware and software levels, artificial intelligence (AI) technology has ushered in a peak of development. It mines key information from massive amounts of data to achieve various applications, such as face recognition, voice recognition, data mining, etc. However, for scenarios where data privacy is sensitive, such as patient information in hospitals and customer information in banks, data is usually difficult to obtain, commonly known as information islands. If the existing artificial intelligence training method is still used, it is difficult to obtain effective results due to insufficient data.

Federated Learning (FL) proposed by Google is used to solve the problem of information islands. It is mainly aimed at businesses that are sensitive to data privacy and security, as well as upcoming scenarios such as autonomous driving and the Internet of Things. It distributes model training to several terminals. The terminals do not need to send raw data to the edge server, but instead send model parameters or gradient information. Based on the traditional stochastic gradient descent training method, the average gradient method used in the federated learning scenario can still achieve good learning performance. In the high-reliability and low-latency wireless communication scenarios of 5G, the realization of autonomous driving, intelligent analysis and decision-making of the Internet of Things are inseparable from federated learning.

In traditional federated learning scenarios, since the terminals and servers are connected by wires, the communication overhead and local computing latency can be ignored. However, with the development of mobile communication networks and the rapid growth of mobile smart devices, in order to quickly realize IoT applications and autonomous driving, artificial intelligence model training can be placed at mobile smart terminals, and traditional wired communications can be changed to wireless communications, making it very convenient to join and exit training terminals. Combining federated learning with wireless communication networks is the future development direction.

However, applying wireless communication to the scenario of federated learning will cause many problems. First, the local computing latency increases. Although the computing power of local terminals is constantly increasing and AI models can be deployed, the gap between them and desktop computers or servers is still relatively large, and the computing latency caused by local gradient calculations cannot be ignored. On the other hand, due to the shortage of wireless communication bandwidth resources and the instability of wireless channels, the transmission of a large amount of model gradient information will bring huge communication overhead and cause a large transmission latency.

In order to solve the problem of high training latency caused by local calculation and gradient transmission, local batch data processing and gradient compression technology are applied in the model training process. In each round of training interaction, the terminal can calculate the gradient of the model based on only a part of the data to reduce the latency caused by local calculation. At the same time, during the transmission process, the gradient information is compressed to represent the original gradient information with less data, reducing the communication transmission time. However, batch processing and gradient compression will affect the convergence rate of the model.

Therefore, while controlling the model training delay, the model convergence rate also needs to be considered. How to reasonably adjust the batch size and gradient compression rate to ensure the training delay and improve the convergence rate is an urgent problem to be solved.

Summary of the invention

The purpose of the present invention is to provide a federated learning method and system based on adjusting batch size and gradient compression rate. The federated learning method improves the convergence rate of the model within a specified learning time by adjusting the batch size and gradient compression rate, and does not require the transmission of original data during federated learning, thereby better protecting the privacy and security of users.

In order to achieve the above-mentioned purpose, the present invention provides the following technical solutions:

In a first aspect, a federated learning method based on adjusting a batch size and a gradient compression rate is provided, wherein a system for implementing the federated learning includes an edge server and multiple terminals in wireless communication with the edge server, wherein the terminals perform model learning based on local data, and the federated learning method includes:

The edge server adjusts the batch size and gradient compression rate of the terminal according to the current batch size and gradient compression rate, and in combination with the computing capability of the terminal and the communication capability between the edge server and the terminal, and transmits the adjusted batch size and gradient compression rate to the terminal;

The terminal performs model learning according to the received batch size, and compresses the gradient information obtained by the model learning according to the received extraction compression rate and outputs it to the edge server;

The edge server averages all received gradient information and synchronizes the gradient average value to the terminal;

The terminal updates the model according to the received gradient average value.

In the present invention, in order to reduce the terminal computing time and the demand for device hardware, the local terminal calculates the gradient in batches, that is, a certain batch size is selected for gradient calculation each time. In order to reduce the transmission information from the terminal to the edge server, save communication overhead, and reduce communication time, the terminal needs to compress the gradient information before uploading it.

In a possible implementation, adjusting the batch size and gradient compression rate of the terminal according to the current batch size and gradient compression rate and in combination with the computing capability of the terminal and the communication capability between the edge server and the terminal includes:

(a) Calculate the current learning delay based on the current batch size and gradient compression rate, combined with the computing power of the terminal and the communication capacity between the edge server and the terminal;

(b) comparing the current learning delay with the prescribed learning delay, if the current learning delay is greater than the prescribed learning delay, reducing the batch size and increasing the gradient compression rate; if the current learning delay is less than the prescribed learning delay, increasing the batch size and reducing the gradient compression rate;

(c) Repeat steps (a) and (b) until the current learning delay is equal to the specified learning delay, and the batch size and gradient compression rate corresponding to the current learning delay equal to the specified learning delay are the adjusted batch size and gradient compression rate.

The current learning delay is higher than the specified learning time, that is, the current training cannot be completed on time. At this time, the batch size should be reduced and the compression rate should be increased to save time for local terminal calculation and wireless transmission. The current learning delay is lower than the specified learning time, that is, the current training can be completed, but the convergence rate can be further improved. At this time, the batch size can be appropriately increased and the compression rate of the gradient can be reduced to improve the convergence performance.

In particular, when the learning time is particularly short, that is, the training task is very tight, the batch size of each terminal should be as small as possible and the compression rate should be as large as possible to ensure the learning delay; when the learning time is particularly long, that is, the task has no strict learning time requirements, the batch size of each terminal should be as large as possible and the compression rate should be as small as possible to ensure the best convergence performance.

In step (a), the calculation process of the current learning delay is:

Calculate the time required for the terminal to calculate the gradient with the current batch size based on the terminal's learning ability and batch size;

The time it takes for the gradient information to be compressed and uploaded to the edge server through the wireless channel is calculated based on the communication capacity and the gradient compression rate;

The time required to calculate and average all gradient information to obtain the average gradient information;

Calculate the time it takes for the edge server to send the average gradient information to each terminal;

Calculate the time required for each terminal to update the model after receiving the average gradient information;

The sum of these five parts of time is the current learning delay.

According to the calculation process of the current learning delay, the current learning delay can be changed by adjusting the batch size and the gradient compression rate. When the batch size increases or the gradient compression rate decreases, the current learning delay increases. When the batch size decreases or the gradient compression rate increases, the current learning delay decreases.

In a possible implementation, when compressing the gradient information, the gradient may be quantized using a quantization loss. In this case, the quantization loss is defined as a compression rate. That is, the gradient compression rate includes a compression rate obtained by performing gradient quantization on the gradient information, which is expressed as:

表示二范数的平方。Among them, x represents gradient information, Q(x) represents the quantization function, and c represents the compression rate.

represents the square of the 2-norm.

The convergence rate of the corresponding model training can be expressed as:

Among them, α ₁ , β ₁ are coefficients related to the model, and α ₁ >0, β ₁ >0, B is the sum of the batch sizes of each terminal, C is the average value of the gradient compression rate of each terminal, and S is the number of training steps.

When the gradient compression rate adopts the compression rate obtained by the gradient quantization method, the gradient compression rate is increased or decreased by increasing or decreasing the number of quantization bits adopted by the quantization function.

In a possible implementation, the gradient compression rate includes a compression rate obtained by performing sparse processing on the gradient information, that is, selecting the maximum m gradients in the gradient matrix for transmission. At this time, the transmission amount of the gradient is defined as the compression rate, which is expressed as:

Here, M represents the size of the model.

The convergence rate of the corresponding model training can be expressed as:

Among them, α ₂ , β ₂ are coefficients related to the model, and α ₂ >0, β ₂ >0, B is the sum of the batch sizes of each terminal, C is the average value of the gradient compression rate of each terminal, and S is the number of training steps.

When the gradient compression rate is obtained by using the gradient sparse method, the gradient compression rate is increased or decreased by increasing or decreasing the number of gradients.

In a possible implementation, the gradient average is obtained using the following formula:

Among them, g _k (w) represents the gradient calculated by the terminal, G(w) represents the average gradient, w represents the model parameter, k represents the terminal index, and K represents the total number of terminals.

In one possible implementation, the following formula is used to update the model based on the gradient average:

w _t+1 = w _t -αG(w _t )

Where t is the number of iterations, G( _wt ) represents the average gradient at the tth iteration, and _wt and _wt+1 represent the model parameters of the terminal at the tth and t+1th iterations, respectively.

In a second aspect, a federated learning system based on adjusting batch size and gradient compression rate is provided, comprising an edge server connected to a base communication terminal, a plurality of terminals in wireless communication with the edge server,

The edge server adjusts the batch size and gradient compression rate of the terminal according to the current batch size and gradient compression rate, and in combination with the computing capability of the terminal and the communication capability between the edge server and the terminal, and transmits the adjusted batch size and gradient compression rate to the terminal;

The terminal performs model learning according to the received batch size, and compresses the gradient information obtained by the model learning according to the received extraction compression rate and outputs it to the edge server;

The edge server averages all received gradient information and synchronizes the gradient average value to the terminal;

The terminal updates the model according to the received gradient average value.

It can be seen from the above technical solutions that the federated learning method and system based on adjusting the batch size and gradient compression rate provided in the embodiments of the present application have the following advantages: compared with directly transmitting data, transmitting gradients can fully protect the privacy and security of user data; compressing gradients can reduce the pressure of communication transmission and reduce latency; dynamically adjusting the batch size and compression rate according to the requirements of learning time can improve the convergence rate of the model while ensuring the training time and model training accuracy.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

FIG1 is a schematic diagram of a federated learning system based on a wireless communication network in an embodiment of the present application;

FIG2 is a schematic diagram of a quantization method in gradient compression in an embodiment of the present application;

FIG3 is a flow chart of batch size and compression rate adjustment according to an embodiment of the present application;

FIG4 is a schematic diagram of an overall training interaction of an embodiment provided in the embodiments of the present application;

FIG5 is a schematic diagram of a sparse method in gradient compression in an embodiment of the present application;

FIG6 is a flow chart of adjusting batch size and compression rate according to another embodiment provided in the embodiments of the present application;

FIG. 7 is a schematic diagram of an overall training interaction of another embodiment provided in the embodiments of the present application.

DETAILED DESCRIPTION

In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific implementation methods described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

FIG1 is a federated learning system based on a wireless communication network in an embodiment of the present application. The federated learning system includes an edge server and a terminal connected to a communication terminal such as a base station. The terminal shares uplink wireless channel resources and completes the training of the neural network model together with the edge server based on the training data of the local terminal. The federated learning system can be used to implement a federated learning method based on adjusting the batch size and the gradient compression rate. Specifically, for the calculation delay caused by local calculations, the batch size can be adjusted according to the computing power of the terminal to reduce the calculation delay; on the other hand, for the communication bottleneck problem, the communication overhead can be reduced by reducing the amount of information transmitted, that is, the gradient compression technology. The gradient compression rate is adjusted according to the state of the wireless channel where the terminal is located, thereby reducing the communication delay. In order to meet the training time requirements, the batch size and compression rate adjustment method in the present invention can improve the convergence rate of model training while ensuring the training delay.

Embodiment 1

The federated learning method based on adjusting the batch size and gradient compression rate provided in this embodiment is applicable to the scenario where multiple mobile terminals and an edge server connected to a communication hotspot (such as a base station) jointly train an artificial intelligence model. For other wireless communication technologies, they can work in the same working mode. Therefore, in this embodiment, the situation of mobile communication technology is mainly considered.

In this embodiment, each terminal uses a batch method to perform local calculations, and the gradient compression method is quantization. In particular, the quantization uses fixed-length quantization. The quantization process is shown in Figure 2. The gradient information is quantized and encoded with a certain number of bits, and then transmitted to the edge server as gradient information. When a high quantization bit number is used to represent the gradient, the original gradient information can be retained as much as possible, while also increasing the amount of information transmitted; when a low quantization bit number is used to quantize the gradient, there is a deviation between the quantized gradient information and the original gradient information, but the amount of information transmitted is reduced. At this time, the compression rate of the gradient can be expressed by the quantization error, that is,

Among them, Q(x) represents the quantization function and x represents the gradient.

When gradient quantization is used, the convergence rate of model training can be expressed as:

Among them, α ₁ , β ₁ are coefficients related to the model, and α ₁ >0, β ₁ >0, B is the sum of the batch sizes of each terminal, C is the average value of the gradient compression rate of each terminal, and S is the number of training steps.

In this embodiment, the ultimate goal is to adjust the batch size and compression rate based on the computing power of each terminal and the state of the wireless channel on the basis of satisfying the training delay, so as to improve the convergence rate of model training.

Specifically, the batch size and compression ratio adjustment algorithm is shown in Figure 3, which includes the following parts:

301. Calculate the current training delay, where the training delay includes five parts:

(1) The time required to calculate the gradient locally with batch size b;

(2) The time it takes for the gradient information to be compressed and uploaded to the edge server through the wireless channel;

(3) Therefore, the time required for averaging after the gradient information is aggregated;

(4) The time required for the edge server to send the calculated average gradient information to each terminal;

(5) The time it takes for each terminal to update the model after receiving the gradient information;

302. Since the purpose of this embodiment is to improve the convergence rate while ensuring the training time, it is necessary to compare the current training time with the prescribed training time T ^max , which is divided into three cases to make corresponding changes.

303. When the training time is less than the specified training time, the batch size and the number of quantization bits should be appropriately increased to improve the convergence rate.

304. When the training time is longer than the specified training time, the batch size and the number of quantization bits should be appropriately reduced to complete the training in time and meet the time requirements.

305. When the training time is equal to the specified training time, the batch size and compression rate at this time can be used for training.

This adjustment process is used in the actual training of federated learning. The specific interaction process between the terminal and the base station is shown in Figure 4, which specifically includes the following contents:

401. Initialization: Each terminal needs to upload relevant information, such as computing capability and the status of the wireless channel, to the base station.

402. The base station obtains a batch size b and a gradient compression rate c according to the information uploaded by each terminal using the adjustment method described in FIG. 3 .

403. Each terminal calculates local gradient information.

404. According to the gradient compression rate, that is, the quantization error, a corresponding quantization bit number is obtained, the gradient is quantized and encoded, and the gradient is transmitted using a wireless channel.

405. The base station receives the gradient information, averages the gradients, and sends them to each terminal.

406. Each terminal downloads average gradient information.

407. Each terminal updates the model using the average gradient information.

By using the invented method, the best model performance can be obtained.

Embodiment 2

The adjustment method provided in this embodiment is applicable to the scenario where multiple mobile terminals and an edge server connected to a communication hotspot (such as a base station) jointly train an artificial intelligence model. For other wireless communication technologies, they can work in the same working mode. Therefore, in this embodiment, the mobile communication situation is mainly considered.

In this embodiment, each terminal uses a batch method to perform local calculations, and the gradient compression method is sparse. Specifically, the sparse method is to select some larger gradients for transmission. The sparse process is shown in Figure 5. After the gradient information is sparse, the selected gradient information and its number are transmitted to the edge server. When more gradient information is retained, the loss of gradient information can be reduced, and the amount of information transmitted is also increased; when less gradient information is retained, more gradient information is lost, but the amount of information transmitted is reduced. At this time, the compression rate of the gradient can be expressed as the ratio of the total model to the number of gradients transmitted, that is,

Here, M represents the size of the model.

When gradient sparsification is used, the convergence rate of model training can be expressed as:

Among them, α ₂ , β ₂ are coefficients related to the model, and α ₂ >0, β ₂ >0, B is the sum of the batch sizes of each terminal, C is the average value of the gradient compression rate of each terminal, and S is the number of training steps.

In this embodiment, the ultimate goal is to adjust the batch size and compression rate based on the computing power of each terminal and the state of the wireless channel on the basis of satisfying the training delay, so as to improve the convergence rate of the model training.

Specifically, the batch size and compression ratio adjustment algorithm is shown in Figure 6, which includes the following parts:

601. Calculate the current training delay, where the training delay includes five parts:

(1) The time required to calculate the gradient locally with batch size b;

(2) The time it takes for the gradient information to be compressed and uploaded to the edge server through the wireless channel;

(3) Therefore, the time required for averaging after the gradient information is aggregated;

(4) The time required for the edge server to send the calculated average gradient information to each terminal;

(5) The time it takes for each terminal to update the model after receiving the gradient information;

602. Since the purpose of this embodiment is to improve the convergence rate while ensuring the training time, it is necessary to compare the current training time with the prescribed training time T ^max , which is divided into three cases to make corresponding changes.

603. When the training time is less than the specified training time, the batch size and the number of transmitted gradients should be appropriately increased to improve the convergence rate.

604. When the training time is longer than the specified training time, the batch size and the number of transmitted gradients should be appropriately reduced to complete the training in time and meet the time requirements.

605. When the training time is equal to the specified training time, the batch size and compression rate at this time can be used for training.

This adjustment process is used in the actual training of federated learning. The specific interaction process between the terminal and the base station is shown in Figure 7, which specifically includes the following contents:

701. Initialization: Each terminal needs to upload relevant information, such as computing capability and the status of the wireless channel, to the base station.

702. The base station obtains a batch size b and a gradient compression rate c according to the information uploaded by each terminal using the adjustment method described in FIG. 6 .

703. Each terminal calculates local gradient information.

704. According to the gradient compression rate, the number of gradients to be transmitted is obtained, the gradients are sorted and selected, the largest gradient and its position are selected, and they are transmitted using a wireless channel.

705. The base station receives the gradient information, averages the gradient, and sends it to each terminal.

706. Each terminal downloads average gradient information.

707. Each terminal uses the average gradient information to update the model.

By using the invented method, the best model performance can be obtained.

This federated learning method improves the convergence rate of the model within the specified learning time by adjusting the batch size and gradient compression rate. It does not require the transmission of original data during federated learning, thus better protecting the privacy and security of users.

Embodiment 3

Embodiment 3 provides a federated learning system based on adjusting batch size and gradient compression rate, including an edge server connected to a base communication terminal, a plurality of terminals in wireless communication with the edge server,

The edge server adjusts the batch size and gradient compression rate of the terminal according to the current batch size and gradient compression rate, and in combination with the computing capability of the terminal and the communication capability between the edge server and the terminal, and transmits the adjusted batch size and gradient compression rate to the terminal;

The terminal performs model learning according to the received batch size, and compresses the gradient information obtained by the model learning according to the received extraction compression rate and outputs it to the edge server;

The edge server averages all received gradient information and synchronizes the gradient average value to the terminal;

The terminal updates the model according to the received gradient average value.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific process of implementing the federated learning system and the federated learning method based on adjusting the batch size and the gradient compression rate provided in Example 1 and Example 2 is the same, and reference can be made to the corresponding process in the aforementioned method embodiments, which will not be repeated here.

The federated learning system improves the convergence rate of the model within the specified learning time by adjusting the batch size and gradient compression rate. It does not need to transmit original data during federated learning, thus better protecting the privacy and security of users.

Among them, the wireless communication method mentioned above can be an existing mobile communication network, namely LTE (Long-term Evolution) or 5G network, or a WiFi network.

The processor capability of the edge server mentioned above far exceeds the computing capability of the local terminal and has the ability to independently perform model training. The processor can be a general-purpose central processing unit (CPU), a graphics processing unit (GPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for executing the above-mentioned model training programs.

Among them, the local terminals mentioned above can be modern smart phones, tablets, notebooks, self-driving cars and other mobile terminals that can support model training, equipped with wireless communication systems, and can access mainstream wireless communication networks such as mobile communication networks and WiFi.

The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the protection scope of the present invention.

Claims (9)

1. A federated learning method based on adjusting batch size and gradient compression rate, wherein a system for implementing the federated learning comprises an edge server and a plurality of terminals in wireless communication with the edge server, wherein the terminals perform model learning based on local data, wherein the federated learning method comprises: The edge server adjusts the batch size and gradient compression rate of the terminal according to the current batch size and gradient compression rate, and in combination with the computing capability of the terminal and the communication capability between the edge server and the terminal, including: (a) Calculate the current learning delay based on the current batch size and gradient compression rate, combined with the computing power of the terminal and the communication capacity between the edge server and the terminal; (b) comparing the current learning delay with the prescribed learning delay, if the current learning delay is greater than the prescribed learning delay, reducing the batch size and increasing the gradient compression rate; if the current learning delay is less than the prescribed learning delay, increasing the batch size and reducing the gradient compression rate; (c) repeating steps (a) and (b) until the current learning delay is equal to the specified learning delay, and the batch size and gradient compression rate corresponding to the current learning delay equal to the specified learning delay are the adjusted batch size and gradient compression rate; The edge server transmits the adjusted batch size and gradient compression rate to the terminal; The terminal performs model learning according to the received batch size, and compresses the gradient information obtained by the model learning according to the received extraction compression rate and outputs it to the edge server; The edge server averages all received gradient information and synchronizes the gradient average value to the terminal; The terminal updates the model according to the received gradient average value. 2. The method for federated learning based on adjusting batch size and gradient compression rate according to claim 1, wherein in step (a), the calculation process of the current learning delay is: Calculate the time required for the terminal to calculate the gradient with the current batch size based on the terminal's learning ability and batch size; The time it takes for the gradient information to be compressed and uploaded to the edge server through the wireless channel is calculated based on the communication capacity and the gradient compression rate; The time required to calculate and average all gradient information to obtain the average gradient information; Calculate the time it takes for the edge server to send the average gradient information to each terminal; Calculate the time required for each terminal to update the model after receiving the average gradient information; The sum of these five parts of time is the current learning delay. 3. The federated learning method based on adjusting batch size and gradient compression rate according to claim 1, wherein the gradient compression rate includes a compression rate obtained by gradient quantization of gradient information, expressed as:

Among them, x represents gradient information, Q(x) represents the quantization function, and c represents the compression rate.

represents the square of the two-norm; The convergence rate of the corresponding model training is expressed as:

Among them, α ₁ , β ₁ are coefficients related to the model, and α ₁ >0, β ₁ >0, B is the sum of the batch sizes of each terminal, C is the average value of the gradient compression rate of each terminal, and S is the number of training steps. 4. The method for federated learning based on adjusting batch size and gradient compression rate according to claim 1, wherein the gradient compression rate includes a compression rate obtained by performing sparse processing on gradient information, that is, selecting a maximum of m gradients in a gradient matrix for transmission, and at this time, the transmission amount of the gradient is defined as the compression rate, which is expressed as:

Where M represents the size of the model; The convergence rate of the corresponding model training is expressed as:

Among them, α ₂ , β ₂ are coefficients related to the model, and α ₂ >0, β ₂ >0, B is the sum of the batch sizes of each terminal, C is the average value of the gradient compression rate of each terminal, and S is the number of training steps. 5. The federated learning method based on adjusting batch size and gradient compression rate as described in claim 3 is characterized in that when the gradient compression rate adopts the compression rate obtained by gradient quantization, the gradient compression rate is increased or decreased by increasing or decreasing the number of quantization bits adopted by the quantization function. 6. The federated learning method based on adjusting batch size and gradient compression rate as described in claim 4 is characterized in that when the gradient compression rate adopts the compression rate obtained by gradient sparsification, the gradient compression rate is increased or decreased by increasing or decreasing the number of gradients. 7. The method for federated learning based on adjusting batch size and gradient compression rate according to claim 1, wherein the gradient average is obtained by the following formula:

Among them, g _k (w) represents the gradient calculated by the terminal, G(w) represents the average gradient, w represents the model parameter, k represents the terminal index, and K represents the total number of terminals. 8. The method for federated learning based on adjusting batch size and gradient compression rate according to claim 1, characterized in that the model is updated according to the gradient average using the following formula: w _t+1 = w _t -αG(w _t ) Where t is the number of iterations, G( _wt ) represents the average gradient at the tth iteration, and _wt and _wt+1 represent the model parameters of the terminal at the tth and t+1th iterations, respectively. 9. A federated learning system based on adjusting batch size and gradient compression rate, comprising an edge server connected to a base communication terminal, and a plurality of terminals in wireless communication with the edge server, characterized in that: The edge server adjusts the batch size and gradient compression rate of the terminal according to the current batch size and gradient compression rate, and in combination with the computing capability of the terminal and the communication capability between the edge server and the terminal, including: (a) calculating the current learning delay according to the current batch size and gradient compression rate, and in combination with the computing capability of the terminal and the communication capability between the edge server and the terminal; (b) comparing the current learning delay with the prescribed learning delay, if the current learning delay is greater than the prescribed learning delay, reducing the batch size and increasing the gradient compression rate; if the current learning delay is less than the prescribed learning delay, increasing the batch size and reducing the gradient compression rate; (c) repeating steps (a) and (b) until the current learning delay is equal to the specified learning delay, and the batch size and gradient compression rate corresponding to the current learning delay equal to the specified learning delay are the adjusted batch size and gradient compression rate; The edge server transmits the adjusted batch size and gradient compression rate to the terminal; The terminal performs model learning according to the received batch size, and compresses the gradient information obtained by the model learning according to the received extraction compression rate and outputs it to the edge server; The edge server averages all received gradient information and synchronizes the gradient average value to the terminal; The terminal updates the model according to the received gradient average value.

Images