CN113139663B - Federal edge learning configuration information acquisition method, device, equipment and medium - Google Patents
Federal edge learning configuration information acquisition method, device, equipment and medium Download PDFInfo
- Publication number
- CN113139663B CN113139663B CN202110444355.XA CN202110444355A CN113139663B CN 113139663 B CN113139663 B CN 113139663B CN 202110444355 A CN202110444355 A CN 202110444355A CN 113139663 B CN113139663 B CN 113139663B
- Authority
- CN
- China
- Prior art keywords
- training
- edge
- optimized
- bandwidth allocation
- loss function
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5072—Grid computing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Computing Systems (AREA)
- Mobile Radio Communication Systems (AREA)
Abstract
The embodiment of the disclosure discloses a method, a device, equipment and a medium for acquiring federated edge learning configuration information, wherein the method comprises the following steps: acquiring channel information of a wireless communication channel between edge equipment and an edge server, equipment information of the edge equipment and a global model parameter dimension value of an artificial intelligence model; acquiring pre-training data; and acquiring an optimized quantization level and an optimized bandwidth allocation strategy according to the channel information, the equipment information, the global model parameter dimension value and the pre-training data, wherein when the loss function value of the trained artificial intelligent model obtained by training according to the optimized quantization level and the optimized bandwidth allocation strategy meets the optimal loss function value constraint, the training time required for training the trained artificial intelligent model is the minimum. According to the embodiment of the disclosure, the training time required by training can be shortest on the premise of ensuring the accuracy of the artificial intelligence model obtained by training, so that the training efficiency is higher.
Description
Technical Field
The disclosure relates to the technical field of machine learning, in particular to a method, a device, equipment and a medium for acquiring federated edge learning configuration information.
Background
During the use of mobile devices (mobile phones, tablet computers, etc.), a large amount of local data, such as image data, text data, various types of log data, etc., is generated. In analyzing these local data, the local data may be uploaded to a data center, and an artificial intelligence model for data analysis may be trained with the help of machine learning. In recent years, with the development of technology, the data volume of local data generated on a mobile device is increased dramatically, the training speed is reduced by uploading the local data to a data center and then performing training, and in order to increase the training speed, the training can be performed by edge learning. Edge learning, which is to access local data on edge equipment by an edge server and calculate on the edge server and the edge equipment, and then train an edge artificial intelligence model for data analysis, so that training can be performed without uploading the local data to a data center.
With the development of technology, federal marginal learning has gradually begun to grow in popularity. In the process of training through federal edge learning, the edge server and the edge device carry out multi-turn communication, in each communication turn, the edge device uploads a gradient update vector to the edge server, the edge server carries out aggregation according to the received gradient update vector and updates artificial intelligence global model parameters, the updated artificial intelligence global model parameters are distributed to each edge device to drive iteration of the next turn, and iterative update of an artificial intelligence model can be realized through multi-turn iteration to obtain a federal edge artificial intelligence model for data analysis.
In the course of training through federal edge learning, since the gradient update vector contains more parameters (possibly containing millions to billions of parameters), more wireless resources are occupied when the gradient update vector is transmitted, which results in higher communication cost for training through federal edge learning. In order to reduce the communication cost, the radio resources occupied when the gradient update vector is transmitted are generally reduced by reducing the precision of the gradient update vector to be transmitted, but reducing the precision of the gradient update vector may cause the accuracy of the transmitted gradient update vector to be reduced, thereby making the training efficiency lower.
Disclosure of Invention
In order to solve the problems in the related art, embodiments of the present disclosure provide a method, an apparatus, a device, and a medium for acquiring federated edge learning configuration information.
In a first aspect, an embodiment of the present disclosure provides a method for acquiring federated edge learning configuration information, where the method is applied to an edge server, and the method includes:
acquiring channel information of a wireless communication channel between edge equipment and an edge server, equipment information of the edge equipment and a global model parameter dimensional value of an artificial intelligence model;
acquiring pre-training data, wherein the pre-training data comprises a loss function value and iterative updating times of the artificial intelligent model after being updated through pre-training iteration;
and acquiring an optimized quantization level and an optimized bandwidth allocation strategy according to the channel information, the equipment information, the global model parameter dimension value and the pre-training data, wherein when the loss function value of the trained artificial intelligent model obtained by training according to the optimized quantization level and the optimized bandwidth allocation strategy meets the optimal loss function value constraint, the training time required for training the trained artificial intelligent model is the minimum.
With reference to the first aspect, in a first implementation manner of the first aspect, the obtaining pre-training data includes:
when the iterative updating times of the artificial intelligence model is 0, sending a random quantization series and an initial global model parameter to the edge device;
when the iterative updating times of the artificial intelligence model are larger than 0, sending the current global model parameters of the artificial intelligence model to the edge equipment;
receiving a quantization gradient sent by the edge device, wherein the quantization gradient is obtained by quantizing a local gradient according to a random quantization series by the edge device, and the local gradient is obtained by calculating by the edge device according to an initial global model parameter and local sample data;
aggregating the quantitative gradients, iteratively updating the artificial intelligence model according to the aggregated gradients, and adding 1 to the iterative updating times of the artificial intelligence model;
and when the iterative update times meet the iterative update times constraint condition, sampling the current loss function value and the current iterative update times to acquire pre-training data.
With reference to the first aspect, in a second implementation manner of the first aspect, the channel information includes a rayleigh fading coefficient of the channel and a total bandwidth of the channel, and the device information includes a transmission power of each edge device, a number of edge devices that are trained, a computation frequency of each edge device, and a floating point operand required for each edge device to process one sample.
With reference to the second implementation manner of the first aspect, in a third implementation manner of the first aspect, the obtaining an optimized quantization level and an optimized bandwidth allocation policy according to the channel information, the device information, the global model parameter dimension value, and the pre-training data includes:
when T is d ·N ∈ When the minimum is needed, the quantization level q is determined as the optimized quantization level, and the bandwidth allocation strategy { b } k Determining the optimized bandwidth allocation strategy;
wherein, b k Bandwidth resource, T, occupied for the kth edge device d Time taken for edge device to communicate with edge server in one round, N ∈ Training the number of training rounds required for training the trained artificial intelligent model when the loss function value of the trained artificial intelligent model meets the optimal loss function value constraint;
wherein, T d Is defined as follows:
R k is defined as follows:
k is the number of edge devices to be trained, v k The floating point operand required to compute each sample for the kth edge device, f k For the calculated frequency of the k-th edge device, d is the global model parameter dimension value of the artificial intelligence model, N 0 Power spectral density, p, of white Gaussian noise k The transmission power of the kth edge device is set to 0 as the mean value of Rayleigh fading coefficients
The circularly symmetric complex Gaussian random variable;
N ∈ is defined as follows:
wherein the content of the first and second substances,
calculated by the following formula:
H m for pre-training data obtained from the m-th pre-training, H m Calculated by the following formula:
alpha is calculated by the following formula;
F a for iterative updating of N in the mth pre-training a Value of loss function after the next time, F b For iterative updating of N in the mth pre-training b Value of loss function after the next time, F c After iterative update N in mth pre-training c The loss function value after the next time.
With reference to the third implementation manner of the first aspect, in a fourth implementation manner of the first aspect, the current T is obtained d ·N ∈ Minimum number of quantization levels q and bandwidth allocation strategy b k Comprises steps a1-a3:
a1, fixing a quantization level q, and obtaining the value T by fast binary search of two layers d ·N ∈ Minimum bandwidth divisionConfiguration strategy b k }；
a2, fixed bandwidth allocation strategy b k Get at T by successive convex approximation optimization d ·N ∈ The minimum number of quantization steps q;
a3, circularly executing the step a1 to the step a2, and when the difference of the quantization level numbers q obtained in the last two times is less than or equal to the threshold of the difference of the optimized quantization level numbers and the optimized bandwidth allocation strategy { b obtained in the last two times k When the difference degree of the quantization step number is smaller than or equal to the difference threshold of the optimized bandwidth allocation strategy, determining the quantization step number q obtained at the last time as the optimized quantization step number, and determining the bandwidth allocation strategy { b) obtained at the last time k And determining as an optimized bandwidth allocation strategy.
With reference to the fourth implementation manner of the first aspect, in a fifth implementation manner of the first aspect, the fast binary search is performed on two layers to obtain the value T d ·N ∈ Minimum bandwidth allocation policy b k And (c) comprises steps b1-b5:
b1, calculation according to the following formula
Calculated according to the following formula
b2, in the outer layer binary search, searching T according to the following formula d′ ：
b3, inIn the inner layer binary search, T is added d′ The following equations are substituted and solved:
R k (b k )＝((1+log 2 (q+1))d)/T d′ -(ν k /f k )；
b4, { b obtained from solution k Calculating
When in use
(Times)
When in use
(Times)
b5, circularly executing the steps b2-b4, and solving the { b ] according to the latest step b3 k Obtained by calculation
When the difference between B and B is less than or equal to the bandwidth difference threshold, { B solved according to the last step B3 k Is determined at T d ·N ∈ Minimum bandwidth allocation policy b k }。
With reference to the fourth implementation manner of the first aspect, in a sixth implementation manner of the first aspect, the disclosure provides a fixed bandwidth allocation policy { b } k Get at T by successive convex approximation optimization d ·N ∈ The minimum number of quantization levels q, comprising steps c1-c3:
c1, solving the following convex problem by using a convex optimization solver:
wherein the content of the first and second substances,
is defined as follows:
J k (q (r) ) Is defined as follows:
J′ k (q (r) ) Is defined as follows:
c2, substituting the quantization series obtained by solving into the step c1;
c3, when the difference of the quantization levels obtained by the last two times of solution is less than or equal to the threshold value of the quantization level, determining the difference at T according to the quantization level obtained by the last time of solution d ·N ∈ The number of quantization steps q at the minimum.
In a second aspect, an embodiment of the present disclosure provides an apparatus for acquiring federated edge learning configuration information, where the apparatus is located in an edge server, and the apparatus includes:
the reference information acquisition module is configured to acquire channel information of a wireless communication channel between the edge device and the edge server, device information of the edge device and a global model parameter dimension value of the artificial intelligence model;
the training data acquisition module is configured to acquire pre-training data, and the pre-training data comprises a loss function value and iterative update times of the artificial intelligence model after being updated through pre-training iteration;
and the configuration information acquisition module is configured to acquire an optimized quantization level and an optimized bandwidth allocation strategy according to the channel information, the equipment information, the global model parameter dimension value and the pre-training data, wherein when the loss function value of the trained artificial intelligent model obtained by training according to the optimized quantization level and the optimized bandwidth allocation strategy meets the optimal loss function value constraint, the training time required for training the trained artificial intelligent model is the minimum.
In a third aspect, an embodiment of the present disclosure provides an electronic device, including a memory and a processor; wherein the memory is configured to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method according to any one of the first aspect, the first to sixth implementation manner of the first aspect.
In a fourth aspect, the present disclosure provides a readable storage medium, on which computer instructions are stored, and when executed by a processor, the computer instructions implement the method according to the first aspect, and the first to sixth implementation manners of the first aspect.
The scheme of the embodiment of the disclosure can solve the defect of low training efficiency in federal marginal learning. The method comprises the steps of obtaining pre-training data by obtaining channel information of a wireless communication channel between edge equipment and a server, equipment information of the edge equipment and a global model parameter dimension value of an artificial intelligence model, obtaining an optimized quantization level and an optimized bandwidth allocation strategy according to the channel information, the equipment information, the global model parameter dimension value and the pre-training data, balancing a compromise relation between the quantization level used for training and the optimized bandwidth allocation strategy, and enabling training time required by training to be shortest on the premise that a loss function value of the artificial intelligence model obtained by training according to the optimized quantization level and the optimized bandwidth allocation strategy meets the optimal loss function value constraint, namely on the premise that the artificial intelligence model obtained by training is accurate, so that training efficiency is high.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
Other features, objects, and advantages of the present disclosure will become more apparent from the following detailed description of non-limiting embodiments when taken in conjunction with the accompanying drawings. In the drawings:
FIG. 1 shows a schematic diagram of an application scenario according to an embodiment of the present disclosure;
FIG. 2 illustrates a flow diagram of a federated edge learning method in accordance with an embodiment of the present disclosure;
fig. 3 illustrates a flow chart of a federated edge learning configuration information acquisition method according to an embodiment of the present disclosure;
fig. 4 illustrates a flow chart of a federated edge learning configuration information acquisition method in accordance with an embodiment of the present disclosure;
FIG. 5 illustrates a flow diagram of a federated edge learning method in accordance with an embodiment of the present disclosure;
fig. 6 shows a block diagram of a federal edge learning configuration information acquisition apparatus according to an embodiment of the present disclosure;
FIG. 7 shows a block diagram of an electronic device according to an embodiment of the present disclosure; and
FIG. 8 illustrates a block diagram of a computer system suitable for implementing a federated edge learning configuration information acquisition method in accordance with an embodiment of the present disclosure.
Detailed Description
Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. Furthermore, parts that are not relevant to the description of the exemplary embodiments have been omitted from the drawings for the sake of clarity.
In the present disclosure, it is to be understood that terms such as "including" or "having," etc., are intended to indicate the presence of the disclosed features, numbers, steps, behaviors, components, parts, or combinations thereof, and are not intended to preclude the possibility that one or more other features, numbers, steps, behaviors, components, parts, or combinations thereof may be present or added.
It should be further noted that the embodiments and features of the embodiments in the present disclosure may be combined with each other without conflict. The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
In considering the application scenario of the disclosed embodiments, the inventors conducted research in a federal edge learning scenario.
Fig. 1 shows a schematic diagram of an application scenario according to an embodiment of the present disclosure. It is to be understood that the application scenario illustrated in fig. 1 is merely for illustrating the concepts and principles of the present disclosure, and is not meant to imply that the present disclosure is only applicable to such application scenario. As shown in fig. 1, in one embodiment of the present disclosure, federal edge learning requires multiple rounds of information interaction by the edge server 101 and the edge device 102.
Fig. 2 shows a flowchart of a federal edge learning method according to an embodiment of the present disclosure, which includes steps S101 to S106, as shown in fig. 2.
S101, the edge device registers to the edge server to apply for training.
S102, the edge server sends the global model parameters and the initial gradients to each edge device.
S103, the edge device calculates a local gradient by using the received global model parameters and the local sample data.
And S104, after quantizing the calculated local gradient, the edge device sends the quantized local gradient, namely the gradient update vector, to the edge server.
S105, the edge server carries out global aggregation according to the received quantized gradient, and updates the global model parameters of the artificial intelligence model according to the aggregated gradient.
And S106, when the loss function value of the artificial intelligent model does not meet the threshold constraint condition, the edge server sends the updated global model parameters to the edge device, and the step S103 is executed.
And S107, when the loss function value of the artificial intelligent model meets the threshold constraint condition, finishing training.
As shown in fig. 1 and fig. 2, in the process of federal edge learning, the edge server 101 and the edge device 102 need to perform multiple rounds of information interaction to realize iterative update of the model, that is, in each communication round (i.e., step S101 to step S106), the edge device 102 needs to upload a gradient update vector to the edge server 101, the edge server 101 also needs to distribute updated global model parameters to each edge device 102 to drive the iteration of the next round, and iterative update of the artificial intelligence model can be realized through multiple rounds of iteration to obtain the federal edge artificial intelligence model for data analysis. Because the gradient update vector contains more parameters (possibly containing millions to billions of parameters), more wireless resources are occupied when the gradient update vector is transmitted, so that the communication cost for training through federal marginal learning is higher, and the communication delay is more serious.
In one technical solution, by reducing the precision of the gradient update vector to be transmitted, the wireless resources occupied when the gradient update vector is transmitted can be reduced, but reducing the precision of the gradient update vector will cause the accuracy of the transmitted gradient update vector to be reduced, thereby reducing the training speed.
In order to solve the above problems, the present disclosure provides a method, an apparatus, a device, and a medium for obtaining federal edge learning configuration information.
The scheme of the embodiment of the disclosure can solve the defect of low training efficiency in federal marginal learning. The method comprises the steps of obtaining pre-training data by obtaining channel information of a wireless communication channel between edge equipment and a server, equipment information of the edge equipment and a global model parameter dimension value of an artificial intelligence model, obtaining an optimized quantization level and an optimized bandwidth allocation strategy according to the channel information, the equipment information, the global model parameter dimension value and the pre-training data, balancing a compromise relation between the quantization level used for training and the optimized bandwidth allocation strategy, and enabling training time required by training to be shortest on the premise of ensuring that the artificial intelligence model obtained through training is accurate, wherein the loss function value of the artificial intelligence model obtained through training according to the optimized quantization level and the optimized bandwidth allocation strategy meets the optimal loss function value constraint.
Fig. 3 is a flowchart illustrating a federal edge learning configuration information acquisition method according to an embodiment of the present disclosure, which is applied to an edge server, and as shown in fig. 3, the method includes steps S110 to S130.
In step S110, channel information of a wireless communication channel between the edge device and the edge server, device information of the edge device, and a global model parameter dimension value of the artificial intelligence model are obtained.
In step S120, pre-training data is acquired.
The pre-training data comprises a loss function value and iterative updating times of the artificial intelligent model after being updated through pre-training iteration;
in step S130, an optimized quantization level and an optimized bandwidth allocation strategy are obtained according to the channel information, the device information, the global model parameter dimension value, and the pre-training data.
When the loss function value of the trained artificial intelligence model obtained by training according to the optimized quantization level and the optimized bandwidth allocation strategy meets the optimal loss function value constraint, the training time required by training the trained artificial intelligence model is the minimum.
According to the technical scheme provided by the embodiment of the disclosure, the channel information of the wireless communication channel between the edge device and the edge server can be transmitted to the edge device and the edge server
The device information of the edge device and the global model parameter dimension value of the artificial intelligence model can be matched
And acquiring an optimized quantization level and an optimized bandwidth allocation strategy according to the channel information, the equipment information, the global model parameter dimension value and the pre-training data, wherein the loss function value of the artificial intelligent model obtained by training according to the optimized quantization level and the optimized bandwidth allocation strategy meets the optimal loss function value constraint, namely, on the premise of ensuring the accuracy of the artificial intelligent model obtained by training, the training time required by training is shortest, so that the training efficiency is higher.
In the scheme of the embodiment of the disclosure, channel information can reflect the condition of a wireless communication channel between edge devices and an edge server during communication, device information can reflect the capability of the edge devices in processing data and the capability of sending data, pre-training data can reflect the difference between local data on different edge devices, an optimized quantization level and an optimized bandwidth allocation strategy are obtained according to the channel information, the device information, a global model parameter dimension value and the pre-training data, the compromise relationship between the quantization level and the optimized bandwidth allocation strategy used by training can be balanced, and a loss function value of an artificial intelligence model obtained by training meets the constraint of an optimal loss function value, that is, on the premise of ensuring that the artificial intelligence model obtained by training is relatively accurate, the training time required by training is shortest, so that the training efficiency is relatively high.
In an embodiment of the present disclosure, as shown in fig. 4, fig. 4 shows a flowchart of a federated edge learning configuration information acquisition method according to an embodiment of the present disclosure, and step S120 may be implemented by steps S121-S125:
in step S121, when the iterative update number of the artificial intelligence model is 0, the random quantization series and the initial global model parameter are sent to the edge device.
In step S122, when the iterative update time of the artificial intelligence model is greater than 0, the current global model parameter of the artificial intelligence model is sent to the edge device.
After receiving the global model parameters, the edge device calculates local gradients by using the received global model parameters and local samples, quantizes the calculated local gradients, and sends the quantized gradients, namely, the quantified speaking gradients to the edge server;
in step S123, the quantization gradient transmitted by the edge device is received.
The quantization gradient is obtained by quantizing the local gradient according to the random quantization series by the edge device, and the local gradient is obtained by calculating the edge device according to the initial global model parameter and the local sample data.
In step S124, the quantization gradients are aggregated, the artificial intelligence model is iteratively updated according to the aggregated gradients, and 1 is added to the iterative update times of the artificial intelligence model.
In step S125, when the iterative update times satisfies the iterative update times constraint condition, the current loss function value and the current iterative update times are sampled to obtain pre-training data.
For example, the constraint condition of the iteration updating times is that the iteration updating times are N a Sub, N b Sub sum N c Second, therefore the number of iterative updates is N a The loss function value F is collected in times a At iteration update times of N b The loss function value F is collected in times b At iteration update times of N c Time-dependent acquisition of loss function value F c 。
Through steps S121 to S125, the pre-training data may be obtained, so that the pre-training data can reflect the degree of heterogeneity between local data on different edge devices, and it is helpful to obtain an optimized quantization level and an optimized bandwidth allocation policy according to the pre-training data, channel information, device information, and global model parameter dimension values, and balance the compromise relationship between the quantization level used in training and the optimized bandwidth allocation policy.
In one embodiment of the disclosure, the channel information includes rayleigh fading coefficients of the channel and total bandwidth of the channel, and the device information includes transmit power of each edge device, the number of edge devices added to training, the computation frequency of each edge device, and the floating point operands required for processing one sample by each edge device.
The method for acquiring the rayleigh fading coefficient of the channel may be as follows: the edge server sends a pilot signal sending instruction to the edge device, so that the edge device sends a pilot signal to the edge server in response to the pilot signal sending instruction, and the edge server performs analysis according to the pilot signal and a communication protocol to estimate the Rayleigh fading coefficient.
The total bandwidth of the channel may be obtained by: the edge server reads the bandwidth setting file stored in advance and obtains the total bandwidth of the channel according to the reading result.
The device information may be acquired in the following manner: and the edge server sends a device information uploading instruction to the edge device, so that the edge device responds to the device information uploading instruction to send the device information to the edge server.
In an embodiment of the present disclosure, obtaining an optimized quantization level and an optimized bandwidth allocation strategy according to channel information, device information, a global model parameter dimension value, and pre-training data may be implemented by:
when T is d ·N ∈ When the minimum is needed, the quantization level q is determined as the optimized quantization level, and the bandwidth allocation strategy { b } k Determining the optimized bandwidth allocation strategy.
Wherein, b k Bandwidth resource, T, occupied for the kth edge device d Time taken for edge device to communicate with edge server in one round, N ∈ Training the number of training rounds required for training the trained artificial intelligent model when the loss function value of the trained artificial intelligent model meets the optimal loss function value constraint;
T d is defined as follows:
R k is defined as follows:
k is the number of edge devices to be trained, v k The floating point operand required to compute each sample for the kth edge device, f k For the calculated frequency of the k-th edge device, d is the global model parameter dimension value of the artificial intelligence model, N 0 Power spectral density, p, of white Gaussian noise k The transmission power of the kth edge device is set to 0 as the mean value of Rayleigh fading coefficients
Is a circularly symmetric complex gaussian random variable.
N ∈ Is defined as follows:
wherein, the first and the second end of the pipe are connected with each other,
calculated by the following formula:
H m for pre-training data obtained from the m-th pre-training, H m Calculated by the following formula:
alpha is calculated by the following formula;
F a for iterative updating of N in the mth pre-training a Value of loss function after the next time, F b Iteratively updating the after-N for the mth pre-training b Value of loss function after the next time, F c After iterative update N in mth pre-training c The loss function value after the next time.
V is k /f k The time of the local gradient is calculated once for the k-th edge device. When the equivalence order is q, the edge device needs to be (1 + log) 2 (q + 1)) d bits to encode the quantized local gradient to be transmitted, thus (1 + log 2 (q+1))d/R k Time, R, for the kth edge device to upload quantized local gradients in the last communication round k For the k-th edge device to the edge serverEach state experiences a transmission capacity. Because the edge server adopts synchronous aggregation when the quantized gradient is subjected to global aggregation, namely the server starts aggregation after receiving the quantized local gradients sent by all the edge devices, a communication turn of training starts from the moment that the edge server sends global model parameters to all the edge devices to the moment that the last edge server receives the quantized local gradients sent by the last edge device, namely the time occupied by the communication turn is v k /f k +(1+log 2 (q+1))d/R k 。
Get when T d ·N ∈ Minimum number of quantization levels q and bandwidth allocation strategy b k Can be a pair T d ·N ∈ Solving to obtain T d ·N ∈ Minimum number of quantization levels q and bandwidth allocation strategy b k T can also be obtained by exhaustion d ·N ∈ Minimum number of quantization levels q and bandwidth allocation strategy b k }。
In addition, the edge server sends the model parameters to the edge device, which occupies all bandwidth resources, and the edge server has higher transmission power than the edge device, so that the communication time of sending the model parameters by the edge server can be ignored compared with the communication time of uploading quantized local gradient by the edge device.
In one embodiment of the present disclosure, when T d ·N ∈ When the minimum is needed, the quantization level q is determined as the optimized quantization level, and the bandwidth allocation strategy { b } k Determining the optimal bandwidth allocation strategy can be realized through steps a1-a3:
in step a1, the quantization series q is fixed, and the quantization series q is obtained by fast binary search of two layers at T d ·N ∈ Minimum bandwidth allocation policy b k }。
In step a2, a fixed bandwidth allocation policy b k Get at T by successive convex approximation optimization d ·N ∈ The number of quantization steps q at the minimum.
In step a3, steps a1-a 2 are performed in a loop, when recentlyThe difference of the quantization level numbers q obtained twice is less than or equal to the threshold of the difference of the optimized quantization level numbers, and the optimized bandwidth allocation strategy b obtained twice most recently k Determining the quantization series q obtained last time as T when the difference degree of the quantization series q is less than or equal to the difference threshold of the optimized bandwidth allocation strategy d ·N ∈ The smallest quantization level q, and the most recently obtained bandwidth allocation policy b k Determine when T d ·N ∈ Minimum bandwidth allocation policy b k }。
The fixed quantization series q is obtained by fast binary search of two layers at T d ·N ∈ Minimum bandwidth allocation policy b k And fixed bandwidth allocation policy b k Get at T by successive convex approximation optimization d ·N ∈ The minimum quantization level q can reduce the acquisition bandwidth allocation strategy b k And the difficulty of quantizing the number of levels q. By performing steps a1-a 2 in a loop, the acquired bandwidth allocation policy b can be made k And continuously approaching the optimized bandwidth allocation strategy, and continuously approaching the quantization level q to the optimized quantization level. When the difference between the two latest quantization levels q is less than or equal to the threshold of the difference between the two optimized quantization levels, it can be understood that the speed of the quantization level q approaching the optimized quantization level is too slow and the significance of the continuous approximation is not great in the process of executing the step a1 to the step a2 for the last time. Similarly, the optimal bandwidth allocation strategy b obtained twice recently k When the difference degree of the bandwidth allocation policies is smaller than or equal to the difference threshold of the optimized bandwidth allocation policies, it can be understood that the bandwidth allocation policies { b ] are enabled in the process of executing the steps a1 to a2 for the last time k The speed of approaching the optimal bandwidth allocation strategy is too slow, and the significance of continuing to approach is not great.
In one embodiment of the present disclosure, fast binary search at T is obtained by two layers d ·N ∈ Minimum bandwidth allocation policy b k } can be achieved by steps b1-b5:
b1, calculation according to the following formula
Calculated according to the following formula
b2, in the outer layer binary search, searching T according to the following formula d′ ：
b3, in the inner layer binary search, the T is searched d′ The following equations are substituted and solved:
R k (b k )＝((1+log 2 (q+1))d)/T d′ -(ν k /f k )；
b4, { b obtained from solution k Calculating
When in use
(Times)
When in use
(Times)
b5, circularly executing the steps b2-b4, and solving the { b ] according to the last step b3 k Obtained by calculation
When the difference between B and B is less than or equal to the bandwidth difference threshold, { B solved according to the last step B3 k Determining said time T d ·N ∈ Minimum bandwidth allocation policy b k }。
In one embodiment of the present disclosure, a fixed bandwidth allocation policy b k Get at T by successive convex approximation optimization d ·N ∈ The minimum number of quantization levels q, can be realized by steps c1-c3:
c1, solving the following convex problem by using a convex optimization solver:
wherein the content of the first and second substances,
is defined as follows:
J k (q (r) ) Is defined as follows:
J′ k (q (r) ) Is defined as follows:
c2, substituting the solved quantization series into the step c1;
c3, when the difference of the quantization levels obtained by the last two times of solution is less than or equal to the threshold value of the quantization level, determining the difference at T according to the quantization level obtained by the last time of solution d ·N ∈ The number of quantization steps q at the minimum.
It should be noted that, when the bandwidth allocation policy is fixed, the quantization bit number is obtained by continuous convex approximation optimization. The basic idea of continuous convex approximation is to obtain a locally optimal solution to the original non-convex problem by solving a series of approximately convex problems.
By introducing intermediate variables
The problem of optimizing the quantization bit number when allocating fixed bandwidth to be solved is described as follows:
introducing a function on the quantization series q:
if the quantization bit number obtained after the r-th iteration is q (r) . Then, the (r + 1) th iteration solves the following convex problem using the convex optimization software package CVX:
wherein the content of the first and second substances,
J′ k (q (r) ) Is J k (q) at q (r) The derivative of (c).
The number of quantization levels obtained by solving the convex problem. Iterating in this way until the quantization series obtained by the iteration of the previous time and the next time is less than a given threshold, and determining the quantization series at T according to the quantization series obtained by the last solving d ·N ∈ The number of quantization steps q at the minimum.
In one embodiment of the present disclosure, as shown in fig. 5, fig. 5 shows a flow chart of a federal edge learning method in accordance with an embodiment of the present disclosure. The federal edge learning method includes steps S1401 to S1419.
In step S1401, the edge device registers with the edge server to apply for joining training.
In step S1402, the edge device transmits a pilot signal and device information to the edge server.
In step S1403, the edge server performs analysis based on the pilot signal and the communication protocol to estimate the rayleigh fading coefficient.
In step S1404, the edge server obtains the transmit power of each edge device, the number of edge devices to be trained, the computation frequency of each edge device, and the floating point operand required for each edge device to process one sample, according to the device information.
In step S1405, the edge server reads the bandwidth setting file stored in advance, and obtains the total bandwidth of the channel according to the reading result.
In step S1406, when the number of iterative updates of the artificial intelligence model is 0, the edge server sends the random quantization progression and the initial global model parameters to the edge device.
In step S1407, when the iterative update time of the artificial intelligence model is greater than 0, the edge server sends the current global model parameters of the artificial intelligence model to the edge device.
In step S1408, the edge device calculates a local gradient according to the global model parameters sent by the edge server and the local sample data.
In step S1409, the edge device quantizes the local gradient according to the random quantization level and transmits the quantized gradient to the edge server.
In step S1410, the edge server receives the quantization gradients sent by the edge device, aggregates the quantization gradients, iteratively updates the artificial intelligence model according to the aggregated gradients, and adds 1 to the iterative update frequency of the artificial intelligence model.
In step S1411, when the iterative update times satisfy the iterative update times constraint condition, the edge server samples the current loss function value and the current iterative update times to obtain pre-training data, where the iterative update times constraint condition includes at least three different iterative update times.
In step S1412, the edge server obtains an optimized quantization level and an optimized bandwidth allocation strategy according to the channel information, the device information, the global model parameter dimension value, and the pre-training data.
In step S1413, the edge server sends the optimized quantization level and the optimized bandwidth allocation policy to the edge device.
In step S1414, the edge server sends global model parameters to the individual edge devices.
In step S1415, the edge device calculates a local gradient using the received global model parameters and local sample data.
In step S1416, the edge device quantizes the obtained local gradient according to the optimized quantization level, and sends the quantized local gradient, i.e., a gradient update vector, to the edge server according to the optimized bandwidth allocation policy.
In step S1417, the edge server performs global aggregation according to the received quantized gradient, and updates a global model parameter of the artificial intelligence model according to the aggregated gradient.
In step S1418, when the loss function value of the artificial intelligence model does not satisfy the threshold constraint condition, the edge server sends the updated global model parameters to the edge device, and performs step 1416.
In step S1429, when the loss function value of the artificial intelligence model satisfies the threshold constraint condition, the training is ended.
In the scheme of the embodiment of the disclosure, channel information can reflect the condition of a wireless communication channel between edge devices and an edge server during communication, device information can reflect the capability of the edge devices in processing data and the capability of sending data, pre-training data can reflect the difference between local data on different edge devices, an optimized quantization level and an optimized bandwidth allocation strategy are obtained according to the channel information, the device information, a global model parameter dimension value and the pre-training data, the compromise relationship between the quantization level and the optimized bandwidth allocation strategy used by training can be balanced, and a loss function value of an artificial intelligence model obtained by training meets the constraint of an optimal loss function value, that is, on the premise of ensuring that the artificial intelligence model obtained by training is relatively accurate, the training time required by training is shortest, so that the training efficiency is relatively high.
Fig. 6 shows a block diagram of a structure of a federal edge learning configuration information acquisition apparatus according to an embodiment of the present disclosure. The federal edge learning configuration information acquisition device is located in the edge server, and the federal edge learning configuration information acquisition device can be implemented by software, hardware or a combination of the software and the hardware to become part or all of the electronic equipment.
As shown in fig. 6, the federal edge learning configuration information acquisition apparatus 200 includes a reference information acquisition module 210, a training data acquisition module 220, and a configuration information acquisition module 230.
A reference information obtaining module 210 configured to obtain channel information of a wireless communication channel between the edge device and the edge server, device information of the edge device, and a global model parameter dimension value of the artificial intelligence model;
a training data obtaining module 220 configured to obtain pre-training data, where the pre-training data includes a loss function value and an iterative update number of the artificial intelligence model after being updated by the pre-training iteration;
and a configuration information obtaining module 230 configured to obtain an optimized quantization level and an optimized bandwidth allocation policy according to the channel information, the device information, the global model parameter dimension value, and the pre-training data, wherein when a loss function value of the trained artificial intelligent model obtained by training according to the optimized quantization level and the optimized bandwidth allocation policy satisfies an optimal loss function value constraint, training time required for training the trained artificial intelligent model is minimum.
According to the embodiment of the disclosure, pre-training data is acquired by acquiring channel information of a wireless communication channel between edge equipment and a server, equipment information of the edge equipment and a global model parameter dimension value of an artificial intelligence model, and an optimized quantization level and an optimized bandwidth allocation strategy are acquired according to the channel information, the equipment information, the global model parameter dimension value and the pre-training data, so that a compromise relationship between the quantization level used for training and the optimized bandwidth allocation strategy can be balanced, and a loss function value of the artificial intelligence model obtained by training according to the optimized quantization level and the optimized bandwidth allocation strategy meets an optimal loss function value constraint, that is, on the premise of ensuring that the artificial intelligence model obtained by training is more accurate, training time required by training is shortest, so that training efficiency is higher.
The present disclosure also discloses an electronic device, and fig. 7 shows a block diagram of the electronic device according to an embodiment of the present disclosure.
As shown in fig. 7, the electronic device 300 comprises a memory 301 and a processor 302, wherein the memory 301 is configured to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor 302 to implement the following method steps:
acquiring channel information of a wireless communication channel between edge equipment and an edge server, equipment information of the edge equipment and a global model parameter dimension value of an artificial intelligence model;
acquiring pre-training data, wherein the pre-training data comprises a loss function value and iterative update times of the artificial intelligent model after being updated through pre-training iteration;
and acquiring an optimized quantization level and an optimized bandwidth allocation strategy according to the channel information, the equipment information, the global model parameter dimension value and the pre-training data, wherein when the loss function value of the trained artificial intelligent model obtained by training according to the optimized quantization level and the optimized bandwidth allocation strategy meets the optimal loss function value constraint, the training time required for training the trained artificial intelligent model is the minimum.
According to an embodiment of the present disclosure, obtaining pre-training data includes:
when the iterative updating times of the artificial intelligence model is 0, sending a random quantization series and an initial global model parameter to the edge device;
when the iteration updating times of the artificial intelligence model are larger than 0, sending the current global model parameters of the artificial intelligence model to the edge device;
receiving a quantization gradient sent by the edge device, wherein the quantization gradient is obtained by quantizing a local gradient according to a random quantization series by the edge device, and the local gradient is obtained by calculating by the edge device according to an initial global model parameter and local sample data;
aggregating the quantitative gradients, iteratively updating the artificial intelligence model according to the aggregated gradients, and adding 1 to the iterative updating times of the artificial intelligence model;
and when the iteration updating times meet the constraint condition of the iteration updating times, sampling the current loss function value and the current iteration updating times to obtain pre-training data.
According to the embodiment of the disclosure, the channel information includes the rayleigh fading coefficient of the channel and the total bandwidth of the channel, and the device information includes the transmission power of each edge device, the number of edge devices added to training, the calculation frequency of each edge device, and the floating point operand required for each edge device to process one sample.
According to the embodiment of the disclosure, acquiring an optimized quantization level and an optimized bandwidth allocation strategy according to channel information, device information, a global model parameter dimension value and pre-training data includes:
when T is d ·N ∈ When the minimum is needed, the quantization level q is determined as the optimized quantization level, and the bandwidth allocation strategy { b } k Determining the optimized bandwidth allocation strategy;
wherein, b k Bandwidth resource, T, occupied for the kth edge device d Time taken for edge device to communicate with edge server in one round, N ∈ Training the number of training rounds required for training the trained artificial intelligent model when the loss function value of the trained artificial intelligent model meets the optimal loss function value constraint;
wherein, T d Is defined as follows:
R k is defined as follows:
k is the number of edge devices to be trained, v k The floating point operand required to compute each sample for the kth edge device, f k For the calculated frequency of the k-th edge device, d is the global model parameter dimension value of the artificial intelligence model, N 0 Power spectral density, p, of white Gaussian noise k The transmission power of the kth edge device is set to 0 as the mean value of Rayleigh fading coefficients
The circularly symmetric complex Gaussian random variable;
N ∈ is defined as follows:
wherein the content of the first and second substances,
calculated by the following formula:
H m for pre-training data obtained from the m-th pre-training, H m Calculated by the following formula:
alpha is calculated by the following formula;
F a for iterative updating of N in the mth pre-training a Value of loss function after the next time, F b For iterative updating of N in the mth pre-training b Value of loss function after the next time, F c After iterative update N in mth pre-training c The loss function value after the next time.
According to an embodiment of the present disclosure, a current T is obtained d ·N ∈ Minimum number of quantization levels q and bandwidth allocation strategy b k And, comprising steps a1-a3:
a1, fixing a quantization series q, and obtaining the value T by two-layer quick binary search d ·N ∈ Minimum bandwidth allocation policy b k }；
a2, fixed bandwidth allocation strategy b k Get at T by successive convex approximation optimization d ·N ∈ The minimum number of quantization steps q;
a3, circularly executing the step a1 to the step a2, and when the difference of the quantization level numbers q obtained in the last two times is less than or equal to the threshold of the difference of the optimized quantization level numbers and the optimized bandwidth allocation strategy obtained in the last two times{b k When the difference degree of the quantization step number is smaller than or equal to the difference threshold of the optimized bandwidth allocation strategy, determining the quantization step number q obtained at the last time as the optimized quantization step number, and determining the bandwidth allocation strategy { b) obtained at the last time k Determine to optimize bandwidth allocation policy.
According to an embodiment of the present disclosure, the fast binary search at T is obtained by two layers d ·N ∈ Minimum bandwidth allocation policy b k And (c) comprises steps b1-b5:
b1, calculation according to the following formula
Calculated according to the following formula
b2, in the outer layer binary search, searching T according to the following formula d′ ：
b3, in the inner layer binary search, the T is searched d′ The following equations are substituted and solved:
R k (b k )＝((1+log 2 (q+1))d)/T d′ -(ν k /f k )；
b4, { b obtained from solution k Calculating
When in use
(Times)
When in use
(Times)
b5, circularly executing the steps b2-b4, and solving the { b ] according to the latest step b3 k Obtained by calculation
When the difference between B and B is less than or equal to the bandwidth difference threshold, { B solved according to the last step B3 k Determining said time T d ·N ∈ Minimum bandwidth allocation policy b k }。
According to an embodiment of the present disclosure, a fixed bandwidth allocation policy { b } k Get at T by successive convex approximation optimization d ·N ∈ The minimum number of quantization levels q, comprising steps c1-c3:
c1, solving the following convex problem by using a convex optimization solver:
wherein the content of the first and second substances,
is defined as follows:
J k (q (r) ) Is defined as follows:
J′ k (q (r) ) Is defined as follows:
c2, substituting the quantization series obtained by solving into the step c1;
c3, when the difference of the quantization levels obtained by the last two times of solution is less than or equal to the threshold value of the quantization level, determining the difference at T according to the quantization level obtained by the last time of solution d The number of quantization steps q when N is minimum.
FIG. 8 illustrates a schematic structural diagram of a computer system suitable for implementing federated edge learning configuration information acquisition in accordance with an embodiment of the present disclosure.
As shown in fig. 8, the computer system 400 includes a processing unit 401 that can execute various methods in the above-described embodiments according to a program stored in a Read Only Memory (ROM) 402 or a program loaded from a storage section 408 into a Random Access Memory (RAM) 403. In the RAM 403, various programs and data necessary for the operation of the system 400 are also stored. The processing unit 401, ROM402, and RAM 403 are connected to each other by a bus 404. An input/output (I/O) interface 405 is also connected to bus 404.
The following components are connected to the I/O interface 405: an input portion 406 including a keyboard, a mouse, and the like; an output section 407 including a display device such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 408 including a hard disk and the like; and a communication section 409 including a network interface card such as a LAN card, a modem, or the like. The communication section 409 performs a communication process via a network such as the internet. A drive 410 is also connected to the I/O interface 405 as needed. A removable medium 411 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 410 as needed, so that a computer program read out therefrom is mounted in the storage section 408 as needed. The processing unit 401 may be implemented as a CPU, a GPU, a TPU, an FPGA, an NPU, or other processing units.
In particular, the above described methods may be implemented as computer software programs according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program comprising program code for performing the above-described method. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 409 and/or installed from the removable medium 411.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units or modules described in the embodiments of the present disclosure may be implemented by software or by programmable hardware. The units or modules described may also be provided in a processor, and the names of the units or modules do not in some cases constitute a limitation of the units or modules themselves.
As another aspect, the present disclosure also provides a computer-readable storage medium, which may be a computer-readable storage medium included in the electronic device or the computer system in the above embodiments; or it may be a separate computer readable storage medium not incorporated into the device. The computer readable storage medium stores one or more programs for use by one or more processors in performing the methods described in the present disclosure.
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention in the present disclosure is not limited to the specific combination of the above-mentioned features, but also encompasses other embodiments in which any combination of the above-mentioned features or their equivalents is possible without departing from the inventive concept. For example, the above features and the technical features disclosed in the present disclosure (but not limited to) having similar functions are replaced with each other to form the technical solution.
Claims (7)
1. A method for acquiring federated edge learning configuration information is applied to an edge server, and the method comprises the following steps:
acquiring channel information of a wireless communication channel between edge equipment and an edge server, equipment information of the edge equipment and a global model parameter dimension value of an artificial intelligence model;
acquiring pre-training data, wherein the pre-training data comprises a loss function value and iterative updating times of the artificial intelligence model after being updated through pre-training iteration;
obtaining an optimized quantization level and an optimized bandwidth allocation strategy according to the channel information, the equipment information, the global model parameter dimension value and the pre-training data, wherein when a loss function value of a trained artificial intelligent model obtained by training according to the optimized quantization level and the optimized bandwidth allocation strategy meets an optimal loss function value constraint, the training time required for training the trained artificial intelligent model is minimum;
the channel information comprises a Rayleigh fading coefficient of the channel and the total bandwidth of the channel, and the equipment information comprises the sending power of each edge equipment, the number of the edge equipment added with training, the calculation frequency of each edge equipment and a floating point operand required by each edge equipment for processing a sample;
the obtaining an optimized quantization level and an optimized bandwidth allocation strategy according to the channel information, the device information, the global model parameter dimension value and the pre-training data includes:
when T is d ·N ∈ When the minimum is needed, the quantization level q is determined as the optimized quantization level, and the bandwidth allocation strategy { b } k Determining the optimal bandwidth allocation strategy;
wherein, b k Bandwidth resource, T, occupied for the kth edge device d Time occupied by the edge device and the edge server for one communication round, N ∈ Training the number of training rounds required for training the trained artificial intelligent model when the loss function value of the trained artificial intelligent model meets the optimal loss function value constraint;
T d is defined as follows:
R k is defined as follows:
k is the number of edge devices to be trained, v k The floating point operand required to compute each sample for the kth edge device, f k For the calculated frequency of the k-th edge device, d is the global model parameter dimension value of the artificial intelligence model, N 0 Power spectral density, p, of white Gaussian noise k The rayleigh fading coefficient is the mean value of 0 and the variance is
The circularly symmetric complex Gaussian random variable;
N ∈ is defined as follows:
wherein the content of the first and second substances,
calculated by the following formula:
H m for pre-training data obtained from the m-th pre-training, H m Calculated by the following formula:
alpha is calculated by the following formula;
F a for the N after the iterative update in the mth pre-training a Value of loss function after the next time, F b For the m-th pre-training iteration updated N b Value of loss function after the next time, F c N after iterative update in the mth pre-training c The loss function value after the next time.
2. The method for obtaining federal edge learning configuration information as claimed in claim 1, wherein the obtaining pre-training data comprises:
when the iterative updating times of the artificial intelligence model is 0, sending a random quantization series and an initial global model parameter to the edge device;
when the iterative updating times of the artificial intelligence model are larger than 0, sending the current global model parameters of the artificial intelligence model to the edge device;
receiving a quantization gradient sent by edge equipment, wherein the quantization gradient is obtained by quantizing a local gradient according to the random quantization series by the edge equipment, and the local gradient is obtained by calculating according to the initial global model parameter and local sample data by the edge equipment;
aggregating the quantitative gradients, iteratively updating the artificial intelligence model according to the aggregated gradients, and adding 1 to the iterative updating times of the artificial intelligence model;
and when the iterative update times meet the iterative update times constraint condition, sampling the current loss function value and the current iterative update times to acquire the pre-training data.
3. The method of claim 1, wherein the current T is a value of a Federal edge learning configuration information d ·N ∈ When the minimum value is obtained, the quantization level q is determined as the optimized quantization level, and the bandwidth allocation strategy { b } k Determining the optimized bandwidth allocation strategy, comprising the steps of a1-a3:
a1, fixing a quantization series q, and obtaining the value T by two-layer quick binary search d ·N ∈ Minimum bandwidth allocation policy b k }；
a2, fixed bandwidth allocation strategy b k Get at T by successive convex approximation optimization d ·N ∈ The minimum number of quantization steps q;
a3, circularly executing the step a1 to the step a2, when the difference of the quantization level numbers q obtained at the last two times is less than or equal to the threshold of the difference of the optimized quantization level numbers,And the optimal bandwidth allocation strategy b obtained in the last two times k When the difference degree of the quantization step number is smaller than or equal to the difference threshold of the optimized bandwidth allocation strategy, determining the quantization step number q obtained at the last time as the optimized quantization step number, and determining the bandwidth allocation strategy { b) obtained at the last time as the optimized quantization step number k Determining the optimal bandwidth allocation policy.
4. The method of claim 1, wherein the obtaining at T is performed by two-level fast binary search d ·N ∈ Minimum bandwidth allocation policy b k And (c) comprises steps b1-b5:
b1, calculation according to the following formula
Calculated according to the following formula
b2, in the outer layer binary search, searching T according to the following formula d′ ：
b3, in the inner layer binary search, the T is searched d′ The following equations are substituted and solved:
R k (b k )＝((1+log 2 (q+1))d)/T d′ -(ν k /f k )；
b4, { b obtained from solution k Calculating
When in use
(Times)
When in use
(Times)
b5, circularly executing the steps b2-b4, and solving the { b ] according to the latest step b3 k Obtained by calculation
When the difference between B and B is less than or equal to the bandwidth difference threshold, solving out { B' according to the last step B3 k Determining said time T d ·N ∈ Minimum bandwidth allocation policy b k }。
5. The utility model provides a bang edge learning configuration information acquisition device which characterized in that includes:
the system comprises a reference information acquisition module, a parameter dimension value acquisition module and a parameter dimension value acquisition module, wherein the reference information acquisition module is configured to acquire channel information of a wireless communication channel between edge equipment and an edge server, the equipment information of the edge equipment and a global model parameter dimension value of an artificial intelligence model;
a training data acquisition module configured to acquire pre-training data including a loss function value and an iterative update number of the artificial intelligence model after being updated by a pre-training iteration;
a configuration information obtaining module configured to obtain an optimized quantization level and an optimized bandwidth allocation strategy according to the channel information, the device information, the global model parameter dimension value, and the pre-training data, wherein when a loss function value of a trained artificial intelligence model obtained by training according to the optimized quantization level and the optimized bandwidth allocation strategy satisfies an optimal loss function value constraint, training time required for training the trained artificial intelligence model is minimum;
the channel information comprises a Rayleigh fading coefficient of the channel and a total bandwidth of the channel, and the equipment information comprises the sending power of each edge equipment, the number of the edge equipment added with training, the calculation frequency of each edge equipment and a floating point operand required by each edge equipment for processing a sample;
the obtaining an optimized quantization level and an optimized bandwidth allocation strategy according to the channel information, the device information, the global model parameter dimension value and the pre-training data includes:
when T is d ·N ∈ When the minimum value is obtained, the quantization level q is determined as the optimized quantization level, and the bandwidth allocation strategy { b } k Determining the optimized bandwidth allocation strategy;
wherein, b k Bandwidth resource, T, occupied for the kth edge device d Time occupied by the edge device and the edge server for one communication round, N ∈ The number of training rounds required for training the trained artificial intelligent model when the loss function value of the trained artificial intelligent model meets the optimal loss function value constraint;
T d is defined as follows:
R k is defined as follows:
k isNumber of edge devices to be trained v k The floating point operand required to compute each sample for the kth edge device, f k For the calculated frequency of the k-th edge device, d is the global model parameter dimension value of the artificial intelligence model, N 0 Power spectral density, p, of white Gaussian noise k The rayleigh fading coefficient is the mean value of 0 and the variance is
The circularly symmetric complex Gaussian random variable;
N ∈ is defined as follows:
wherein the content of the first and second substances,
calculated by the following formula:
H m for pre-training data obtained from the m-th pre-training, H m Calculated by the following formula:
alpha is calculated by the following formula;
F a for the N after the iterative update in the mth pre-training a Value of loss function after the next time, F b For the mth preliminaryAfter iterative update N in training b Value of loss function after the next time, F c N after iterative update in the mth pre-training c The loss function value after the next time.
6. An electronic device comprising a memory and a processor; wherein the memory is configured to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method of any one of claims 1-4.
7. A readable storage medium having stored thereon computer instructions, characterized in that the computer instructions, when executed by a processor, implement the method of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110444355.XA CN113139663B (en) | 2021-04-23 | 2021-04-23 | Federal edge learning configuration information acquisition method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110444355.XA CN113139663B (en) | 2021-04-23 | 2021-04-23 | Federal edge learning configuration information acquisition method, device, equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113139663A CN113139663A (en) | 2021-07-20 |
| CN113139663B true CN113139663B (en) | 2023-01-20 |
Family
ID=76812490
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110444355.XA Active CN113139663B (en) | 2021-04-23 | 2021-04-23 | Federal edge learning configuration information acquisition method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113139663B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113660112B (en) * | 2021-07-23 | 2023-11-24 | 广州大学 | Federal learning bandwidth allocation method, system, computer equipment and medium |
| CN114040425B (en) * | 2021-11-17 | 2024-03-15 | 中电信数智科技有限公司 | Resource allocation method based on global resource utility rate optimization |
| CN114363923B (en) * | 2021-11-30 | 2024-03-26 | 山东师范大学 | Industrial Internet of things resource allocation method and system based on federal edge learning |
| CN116264684A (en) * | 2021-12-10 | 2023-06-16 | 华为技术有限公司 | Artificial intelligence AI model training method and device in wireless network |
| CN116579438A (en) * | 2022-01-29 | 2023-08-11 | 华为技术有限公司 | Information processing method and device |
| WO2023197100A1 (en) * | 2022-04-11 | 2023-10-19 | Qualcomm Incorporated | Loss reporting for distributed training of a machine learning model |
| CN115329990B (en) * | 2022-10-13 | 2023-01-20 | 合肥本源物联网科技有限公司 | Asynchronous federated learning acceleration method based on model segmentation under edge computing scene |
| CN115830709A (en) * | 2022-11-23 | 2023-03-21 | 深圳市大数据研究院 | Action recognition method based on federal edge learning, server and electronic equipment |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110505099A (en) * | 2019-08-28 | 2019-11-26 | 重庆邮电大学 | A kind of service function chain dispositions method based on migration A-C study |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109951438B (en) * | 2019-01-15 | 2020-11-20 | 中国科学院信息工程研究所 | Communication optimization method and system for distributed deep learning |
| CN110889510B (en) * | 2019-12-05 | 2022-06-07 | 武汉大学 | Online scheduling method and device for distributed machine learning task |
| CN111447083B (en) * | 2020-03-10 | 2022-10-21 | 中国人民解放军国防科技大学 | Federal learning framework under dynamic bandwidth and unreliable network and compression algorithm thereof |
| CN111901829B (en) * | 2020-07-10 | 2022-05-10 | 江苏智能交通及智能驾驶研究院 | Wireless federal learning method based on compressed sensing and quantitative coding |
-
2021
- 2021-04-23 CN CN202110444355.XA patent/CN113139663B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110505099A (en) * | 2019-08-28 | 2019-11-26 | 重庆邮电大学 | A kind of service function chain dispositions method based on migration A-C study |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113139663A (en) | 2021-07-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113139663B (en) | Federal edge learning configuration information acquisition method, device, equipment and medium | |
| US20190279088A1 (en) | Training method, apparatus, chip, and system for neural network model | |
| CN110956202B (en) | Image training method, system, medium and intelligent device based on distributed learning | |
| CN112925637A (en) | Load balancing device and method for edge operation network | |
| CN114861790B (en) | Method, system and device for optimizing federal learning compression communication | |
| CN114663848A (en) | Knowledge distillation-based target detection method and device | |
| CN111353620A (en) | Method, device and equipment for constructing network point component prediction model and storage medium | |
| CN114781650B (en) | Data processing method, device, equipment and storage medium | |
| CN112819157A (en) | Neural network training method and device and intelligent driving control method and device | |
| CN112465737B (en) | Image processing model training method, image processing method and image processing device | |
| CN114239799A (en) | Efficient target detection method, device, medium and system | |
| CN113592033A (en) | Oil tank image recognition model training method, oil tank image recognition method and oil tank image recognition device | |
| US20230214638A1 (en) | Apparatus for enabling the conversion and utilization of various formats of neural network models and method thereof | |
| CN116384513A (en) | Yun Bianduan collaborative learning system and method | |
| CN111898061B (en) | Method, apparatus, electronic device and computer readable medium for searching network | |
| CN115392348A (en) | Federal learning gradient quantification method, high-efficiency communication Federal learning method and related device | |
| US11689693B2 (en) | Video frame interpolation method and device, computer readable storage medium | |
| CN115131633A (en) | Model migration method and device and electronic equipment | |
| CN111626044B (en) | Text generation method, text generation device, electronic equipment and computer readable storage medium | |
| CN110782017B (en) | Method and device for adaptively adjusting learning rate | |
| CN112081565B (en) | Method and device for determining volume sweep coefficient of water-drive reservoir | |
| CN111582456A (en) | Method, apparatus, device and medium for generating network model information | |
| CN114398986A (en) | Training method and device for hyper network, computer readable medium and electronic equipment | |
| CN117636100B (en) | Pre-training task model adjustment processing method and device, electronic equipment and medium | |
| CN117671364A (en) | Model processing method and device for image recognition, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |