WO2021169577A1

WO2021169577A1 - Wireless service traffic prediction method based on weighted federated learning

Info

Publication number: WO2021169577A1
Application number: PCT/CN2020/140916
Authority: WO
Inventors: 张海霞; 张传亭; 袁东风; 郭帅帅; 周晓天
Original assignee: 山东大学
Priority date: 2020-02-27
Filing date: 2020-12-29
Publication date: 2021-09-02
Also published as: CN111369042A; CN111369042B

Abstract

A wireless service traffic prediction method based on weighted federated learning. The method comprises: a control center pushing a plurality of pre-trained models to base station sides; the base station sides performing model training according to local data, and transmitting the trained models to the control center; the control center fusing the models according to a weighting rule and feeding the fused models back to the base stations, wherein in the weighting rule, more weight is given to local models; and the base stations predicting a wireless service traffic at a future moment according to obtained final models. According to the provided wireless service traffic prediction method, on the basis of a model aggregation policy of the control center, a weighted aggregation rule is used to replace an average policy, such that the phenomenon of inaccurate prediction caused by data heterogeneity can be avoided, thereby improving the overall prediction precision of distributed wireless service traffic prediction.

Description

A wireless service flow prediction method based on weighted federated learning

Technical field

The invention relates to a wireless service flow prediction method based on weighted federated learning, and belongs to the technical field of communication networks and artificial intelligence.

Background technique

The traditional central wireless service traffic prediction method needs to collect large-scale, scattered service data at different nodes to the central node, and then centrally process, train and predict these data. Subsequently, according to the prediction result, the core network makes dynamic adjustments to the base station through the control unit, such as increasing or decreasing the number of baseband processing units to adjust the service capability of the base station.

However, due to the limited bandwidth of data transmission and the problem of data privacy, the transmission of data to the cloud center takes up a lot of resources, causing network congestion; in addition, as users' requirements for data privacy protection continue to increase, especially in general data protection After the release of the General Data Protection Regulation (GDPR), higher requirements for privacy protection have been put forward. Transmitting data to the cloud center increases the possibility of uncontrollable data. These factors promote the development of future predictive models in a distributed, localized, and lightweight direction.

Federated learning belongs to a specific distributed learning algorithm. The local base station trains a local model based on its own data; then it only needs to send this model to the cloud control center, instead of sending a huge data ontology; after the cloud control center receives all the models, it merges the models and sends them to Base station: After receiving the global model, the base station continues to train this model. After the process is repeated for a certain cycle, the final prediction model is obtained.

However, in traditional federated learning algorithms, only the averaging operation is performed on the models, ignoring the differences between the models. The location of the base station is different, and the mobile and communication behaviors of users in the coverage area are also different. This results in a large difference in data. The traditional simple average cannot accurately capture the traffic patterns of different base stations, so the prediction effect is not accurate. . Therefore, there is an urgent need to develop a predictive model that can not only consider model similarity, but also focus on capturing local business traffic patterns.

Summary of the invention

In view of the shortcomings of the prior art, the present invention proposes a wireless service traffic prediction method based on weighted federated learning. In the present invention, after the control center pushes multiple pre-training models to the base station side, the base station side performs model training based on local data. And transmit the trained model to the control center; the control center merges the model according to the weighting rule and feeds it back to the base station, and the weighting rule gives more weight to the local model; the base station learning unit updates the model again based on historical data; the present invention provides The method, in the control center model aggregation strategy, uses weighted aggregation rules to replace the average strategy, which can avoid inaccurate predictions caused by data heterogeneity.

The technical scheme of the present invention is:

A wireless service flow prediction method based on weighted federated learning is used to improve the overall prediction accuracy of distributed wireless service flow prediction. The specific steps are as follows:

(1) Number N base stations in sequence i, i=1, 2,..., N; the control center randomly generates an initialization model M, and copies N models, consisting of N models

Subsequently, the control center will need to train the model

Push to the learning unit of base station i;

(2) The model that the learning unit of base station i needs to train according to the received

Align the model on the existing historical data set

Training is performed to update the parameters; after training, a new local model M′ _{i is obtained} ; the parameters of the model are the weights to be learned, such as the parameters of the long and short memory neural network.

(3) The base station i sends the new local model M′ _i to the control center;

(4) The control center _{performs model fusion on the model {M′ 1} ,..., M′ _N } obtained from the base station according to the weighting strategy, and obtains the updated global model

And the updated global model

Push to base station i;

(5) Repeat steps (2) to (4) to set the number of cycles. When the number of cycles is reached, the cycle ends and the final model is obtained; the parameters are updated each time the cycle is repeated.

(6) The base station predicts the wireless service flow at a future time according to the final model obtained in step (5).

According to the present invention, in the step (4), the specific steps of performing model fusion according to the weighting strategy are: updating the parameters of the global model according to formula (1):

In formula (1),

Represents the global model of the control center after weighted fusion in this round of training; f _agg (·) represents the weighted fusion strategy adopted; α represents the _{proportion of the local model M′ i} for base station i; β represents the addition of the local model The proportion of the sum of M′ _j other than M′ _i ; that is, there are a total of N models, and other models refer to the sum of other N-1 models except for the model with the subscript i. The proportion of the total; the relationship between α and β satisfies α+β=1;

Further preferably, in the step (4), when the parameters of the global model are updated, α>β. When α>β, the local model _{M'i is} given a larger weight to personally capture the current flow pattern of the base station.

Using a weighting strategy instead of an averaging strategy can avoid the inaccurate predictions caused by data heterogeneity, and improve the overall prediction accuracy of distributed wireless service traffic prediction.

According to the present invention, the prediction method preferably further includes: (7) Repeat steps (2)-(6) in a certain period of time to perform periodic global update training on the model; the advantage of this design is that as The continuous accumulation of data sets and the emergence of possible new traffic patterns avoid the current global model of base station i

The ability to predict future moments is weakened; the size of the period can be selected and determined according to the load situation on the base station side to increase the accuracy of the prediction method.

Preferably, the time period is one day or three days or one week;

Preferably, the number of base stations participating in the update training is dynamically changing each time. If the number of base stations currently participating in the training is greater than the set threshold, such as 10%, then the model is globally updated and trained; otherwise, the update is skipped. , And make an update request in the next cycle.

According to the preferred embodiment of the present invention, in step (1), the model that needs to be trained in the control center

Before pushing to the base station learning unit, you need to determine the specific form of the initialization model, generate training sample data sets, test sample data sets, and data standardization. The specific steps include:

1-A. Select the specific form of the initialization model, where the specific form is a linear model or a nonlinear model;

1-B. Divide the historical data set of the base station into a training data set and a test data set, and select the sliding window size p on the training data set and the test data set according to the sliding window mechanism, and generate the training sample data set and the test data set respectively Sample data set;

1-C. For the training sample data set, obtain the minimum value and standard deviation of the flow; for the test sample data set, standardize the data in the training data set and the test data set according to the minimum and standard deviation of the flow.

The function of the training sample data set is to train the model, and the function of the test sample data set is to test the accuracy of the trained model.

According to the present invention, in step (2), the model that needs to be trained is performed on the existing historical data set.

Training to update the parameters, the specific steps include:

2-A. Select an optimization algorithm, the algorithm is any one of stochastic gradient descent method, small batch gradient descent method, and adaptive momentum estimation method (Adam);

2-B. From the training data set, select the corresponding number of samples according to the batch size, and perform gradient calculation; the batch size refers to the number of samples entered during each iteration of training;

2-C, model

The parameters of are updated according to the gradient information of the current sample;

2-D. Repeat steps B and C until the training end conditions are met;

Further preferably, in step 2-A, the algorithm is an adaptive momentum estimation method; the Adam optimization algorithm has the advantage of faster convergence;

In step 2-D, the training end condition needs to meet one of the following conditions: the parameters of the trained model converge; the number of updates of the trained model is greater than the set threshold; the duration of training the entire model is greater than the set threshold.

Preferably, according to the present invention, after the prediction value is obtained in step (6), the prediction method further includes the following steps:

a. The predicted value obtained in step (6) is subjected to a standardized inverse operation to obtain the true scale of the predicted value;

b. Evaluate the prediction performance according to the evaluation index; the evaluation index includes the mean square error MSE and the average absolute error MAE;

c. After the evaluation is completed, the base station stores the newly arrived data in the historical data set. The newly arrived data refers to the newly received wireless service flow data.

The beneficial effects of the present invention are:

1. The wireless traffic prediction algorithm based on weighted federated learning provided by the present invention uses weighted aggregation rules to replace the average strategy in the control center model aggregation strategy, and fully considers the location of different base stations and the movement and communication of users within the coverage area. The difference in behavior takes into account the difference of data; at the same time, the similarity of the model is also taken into account; it can avoid the inaccurate prediction caused by the heterogeneity of data, and improve the overall prediction accuracy of distributed wireless service traffic prediction.

2. The present invention models wireless service traffic prediction as a federated learning problem, and proposes a weighted federated learning prediction algorithm, which avoids network congestion, can better protect privacy, and has the advantages of distributed, localized, and lightweight .

3. By periodically updating the model, the present invention can catch the dynamic change of wireless service flow in time, adjust the learning parameters, and thus has a strong generalization ability.

Description of the drawings

Figure 1 is a schematic diagram of a wireless service traffic prediction system model based on weighted federated learning;

2 is a block diagram of the core flow chart of weighted federated learning training of the present invention;

Figure 3a is a schematic diagram of the comparison of the mean square error between the traditional algorithm and the prediction method provided by the present invention under different numbers of base stations;

Figure 3b is a schematic diagram showing the comparison of the average absolute error between the traditional algorithm and the prediction method provided by the present invention under different numbers of base stations;

FIG. 4 is a schematic diagram of the comparison result between the predicted value provided by Embodiment 1 and the real value and the predicted value of the prior art.

Figure 5 is a schematic diagram of error analysis between the predicted value and the true value of a certain base station.

Detailed ways

The present invention will be further described below in conjunction with the embodiments and the drawings of the specification, but it is not limited thereto.

Example 1

A method for predicting wireless service flow based on weighted federated learning. The system model is shown in Figure 1. The wireless service flow model includes a control center and N base stations.

The base station in Figure 1 shows that in the future communication network, the base station has three functions: network control according to intelligent algorithms, strong computing power, and wireless network access capabilities. "Intelligence" means the deployed machine learning model; "computing" means having strong CPU and GPU computing capabilities, and "accessing" means having wireless access capabilities. Together, these three capabilities can realize the edge intelligence of the future network.

The core process of weighted federated learning training is shown in Figure 2. The model includes N base stations, where N=5, 10, 15, 20. Each base station contains 1448 time sequence points.

The specific steps of the forecasting method include:

Subsequently, the control center will need to train the model

Push to the learning unit of base station i;

In step (1), before the control center pushes the model to be trained to the base station learning unit, it needs to determine the specific form of the initialization model, generate training sample data sets, test sample data sets, and data standardization. The specific steps include:

1-A. Select the specific form of the initialization model. The specific form can be a linear model or a non-linear model; linear models such as logistic regression, and non-linear models such as deep neural networks; due to the complex temporal and spatial characteristics of wireless service traffic, it greatly exceeds the current For the modeling capability of the model, the present invention selects a neural network to capture the pattern of wireless service traffic. The specific form of the initialization model in this embodiment is a long- and short-term memory neural network.

In this embodiment, for the historical data of the base station, the data of the last seven days is selected as the test data set, and the remaining data is used as the training data set. According to the sliding window mechanism, select the window size p=5 to generate 1285 training sample data sets, and a total of 163 test sample data sets;

Align the model on the existing historical data set

Training is performed to update the parameters; after training, a new local model M′ _{i is obtained} ; the updated parameters refer to the parameters to be learned, and in this embodiment are the parameters of the long and short-term memory neural network.

In step (2), the model to be trained on the existing historical data set

Training to update the parameters, the specific steps include:

2-A. Select an optimization algorithm, the algorithm is any one of stochastic gradient descent, small batch gradient descent, and adaptive momentum estimation (Adam); in this embodiment, in step 2-A, the The algorithm is an adaptive momentum estimation method; Adam optimization algorithm has the advantage of faster convergence;

2-C, model

2-D. Repeat steps B and C until the training end conditions are met;

(3) The base station i sends the new local model M′ _i to the control center;

And the updated global model

Push to base station i;

In the step (4), the specific steps of performing model fusion according to the weighting strategy are: updating the parameters of the global model according to formula (1):

In formula (1),

In this embodiment, in the step (4), when the parameters of the global model are updated, α>β. When α>β, the local model _{M'i is} given a larger weight to personally capture the current flow pattern of the base station. In this embodiment, α=0.8, β=0.2.

(5) Repeat steps (2) to (4) to set the number of cycles. When the number of cycles is reached, the cycle ends and the final model is obtained; each cycle will update the model parameters.

(7) Repeat steps (2)-(6) in a certain period of time to perform periodic global update training on the model; the advantage of this design is that due to the continuous accumulation of data sets and the emergence of possible new traffic patterns , Avoiding the current global model of base station i

The time period is one day or three days or one week;

The number of base stations participating in the update training is dynamically changing each time. If the number of base stations currently participating in the training is greater than the set threshold, such as 10%, the model will be globally updated and trained; otherwise, skip this update and the next one Do update requests periodically.

The prediction method also includes the following steps:

The current traditional algorithm is the classic model averaging. The specific steps are as follows: each base station trains a model based on its own historical data, and then sends this model to the cloud control center; after the cloud control center receives all models, it simply averages these models , Get a global model, and send this global model to the base station; according to the received global model, the base station updates the model again based on its own data, and sends the updated model to the base station; repeat the above process, Until the algorithm stops. Then, each base station predicts the future traffic based on the global model sent by the last cloud center.

The prediction performance of the prediction method provided by the present invention is tested and evaluated, and compared with the traditional algorithm and the real flow value. The specific results are as follows:

As shown in Figure 3a and Figure 3b, for different situations where the number of base stations is N, N=5, 10, 15, 20, as the number of base stations increases, the number of samples involved in training continues to increase, regardless of the mean square error or The average absolute error, compared with the prediction result of the traditional algorithm, the error of the prediction method provided by the present invention will gradually decrease, and the weighting strategy can effectively improve the prediction performance.

It can be seen from the comparison of the predicted value of the present invention and the predicted value of the traditional algorithm with the true value in FIG. 4 that the predicted value of the present invention is closer to the true value. precise. And the overall error is much smaller than that of traditional algorithms. Therefore, the wireless service flow prediction scheme proposed by the present invention can effectively improve the prediction performance.

Figure 5 is a cumulative probability distribution diagram of prediction errors. The diagram includes the cumulative probability distribution of prediction errors of the prediction method provided by the present invention and the cumulative probability distribution of prediction errors of traditional algorithms; 50% of the predicted value errors are less than 0.2, while only about 30% of the prediction errors in the traditional algorithm are less than 0.2; the prediction error of the present invention is less than 0.5, accounting for 88%, and the traditional algorithm is 80%. In summary, the prediction method provided by the present invention is superior to the traditional prediction method.

Claims

A wireless service flow prediction method based on weighted federated learning is characterized in that it is used to improve the overall prediction accuracy of distributed wireless service flow prediction. The specific steps are as follows:

(1) Number N base stations in sequence i, i=1, 2,..., N; the control center randomly generates an initialization model M, and copies N models, consisting of N models
Subsequently, the control center will need to train the model
Push to the learning unit of base station i;

(2) The model that the learning unit of base station i needs to train according to the received
Align the model on the existing historical data set
Perform training to update the parameters; after training, a new local model M′ i is obtained ;

(3) The base station i sends the new local model M′ i to the control center;

(4) The control center performs model fusion on the model {M′ 1 ,..., M′ N } obtained from the base station according to the weighting strategy, and obtains the updated global model
And the updated global model
Push to base station i;

(5) Repeat steps (2) to (4) and set the number of cycles. When the number of cycles is reached, the cycle ends and the final model is obtained;

(6) The base station predicts the wireless service flow at a future time according to the final model obtained in step (5).
The method for predicting wireless service traffic based on weighted federated learning according to claim 1, characterized in that, in the step (4), the specific steps of performing model fusion according to the weighting strategy are: performing global operation according to formula (1) The parameter update of the model:

In formula (1),
Represents the global model of the control center after weighted fusion in this round of training; f agg (·) represents the weighted fusion strategy adopted; α represents the proportion of the local model M′ i for base station i; β represents the addition of the local model The proportion of the sum of M′ j other than M′ i ; the relationship between α and β satisfies α+β=1;

Further preferably, in the step (4), when the parameters of the global model are updated, α>β.
The method for predicting wireless service traffic based on weighted federated learning according to claim 1, wherein the predicting method further comprises: (7) repeating steps (2)-(6) in a certain period of time, The model undergoes periodic global update training;

Preferably, the time period is one day or three days or one week.
The method for predicting wireless service traffic based on weighted federated learning according to claim 3, wherein the number of base stations participating in the update training is dynamically changing each time, and if the number of base stations currently participating in the training is greater than the set threshold, Then perform global update training on the model; otherwise, skip this update and make an update request in the next cycle.
The method for predicting wireless service traffic based on weighted federated learning according to claim 1, characterized in that, in step (1), the model to be trained in the control center
Before pushing to the base station learning unit, you need to determine the specific form of the initialization model, generate training sample data sets, test sample data sets, and data standardization. The specific steps include:

1-A. Select the specific form of the initialization model, where the specific form is a linear model or a nonlinear model;

1-B. Divide the historical data set of the base station into a training data set and a test data set, and select the sliding window size p on the training data set and the test data set according to the sliding window mechanism, and generate the training sample data set and the test data set respectively Sample data set;

1-C. For the training sample data set, obtain the minimum value and standard deviation of the flow; for the test sample data set, standardize the data in the training data set and the test data set according to the minimum and standard deviation of the flow.
The method for predicting wireless service traffic based on weighted federated learning according to claim 5, characterized in that, in step (2), the model that needs to be trained is performed on an existing historical data set.
Training to update the parameters, the specific steps include:

2-A. Select an optimization algorithm, the algorithm is any one of stochastic gradient descent method, small batch gradient descent method, and adaptive momentum estimation method;

2-B. From the training data set, select the corresponding number of samples according to the batch size, and perform gradient calculation;

2-C, model
The parameters of are updated according to the gradient information of the current sample;

2-D. Repeat steps B and C until the training end conditions are met;

Further preferably, in step 2-A, the algorithm is an adaptive momentum estimation method;

In step 2-D, the training end condition needs to meet one of the following conditions: the parameters of the trained model converge; the number of updates of the trained model is greater than the set threshold; the duration of training the entire model is greater than the set threshold.
The method for predicting wireless service traffic based on weighted federated learning according to any one of claims 1-6, characterized in that, after the predictive value is obtained in step (6), the predicting method further comprises the following steps:

a. The predicted value obtained in step (6) is subjected to a standardized inverse operation to obtain the true scale of the predicted value;

b. Evaluate the prediction performance according to the evaluation index; the evaluation index includes the mean square error MSE and the average absolute error MAE:

c. After the evaluation is completed, the base station stores the newly arrived data in the historical data set.