CN117786566A - Training method of load prediction model, and load prediction method and device of server - Google Patents

Abstract

The application discloses a training method of a load prediction model, a load prediction method and a device of a server, wherein the model comprises an encoder and a decoder, the encoder comprises a first graph roll-up neural network, a first feedforward neural network, a first multi-head attention mechanism network and a second feedforward neural network, the decoder comprises a second graph roll-up neural network, a third feedforward neural network, a second multi-head attention mechanism network and a fourth feedforward neural network, and the method comprises the following steps: sequentially inputting each graph structure in each training sample into each network in an encoder to obtain a first path network space-time characteristic after convolution calculation of each training sample, sequentially inputting each graph structure in each truth sample into a second graph convolution neural network and a third feedforward neural network to obtain a second path network space characteristic after convolution calculation, sequentially inputting the first path network space-time characteristic and the second path network space characteristic into a second multi-head attention mechanism network and a fourth feedforward neural network, and obtaining a final model through adjustment training.

Description

Training method of load prediction model, and load prediction method and device of server

Technical Field

The application relates to the technical field of cloud computing, in particular to a training method of a load prediction model, a load prediction method of a server and a device.

Background

Cloud computing platforms are one of the important development directions in the current information technology field, and meet the demands of enterprises and individuals for efficient, extensible and reliable computing capabilities by providing elastic resources and flexible service modes. In a cloud computing platform, server load prediction is a key task, and can help a cloud service provider to effectively plan and manage resources, ensure the service quality of users, and simultaneously reduce cost and improve profits.

In the related art, many statistical-based load prediction models have been proposed and applied, such as time series models (e.g., ARIMA (Autoregressive Integrated Moving Average, autoregressive differential moving average) model, SARIMA (Seasonal Autoregressive Integrated Moving Average, seasonal differential autoregressive moving average) model), regression models (e.g., linear regression, support vector machine, etc.), and some deep learning models (e.g., recurrent neural networks, long and short term memory networks, and convolutional neural networks). However, these conventional methods often have difficulty in dealing with complex nonlinear relationships and spatio-temporal dependencies, and thus have certain limitations in terms of prediction accuracy.

Disclosure of Invention

The application provides a training method of a load prediction model, a load prediction method and a load prediction device of a server, which can improve the accuracy of load prediction.

The specific technical scheme is as follows:

in a first aspect, embodiments of the present application provide a training method of a load prediction model, where the load prediction model includes an encoder and a decoder, the encoder includes a first graph roll-up neural network, a first feedforward neural network, a first multi-head attention mechanism network, and a second feedforward neural network, and the decoder includes a second graph roll-up neural network, a third feedforward neural network, a second multi-head attention mechanism network, and a fourth feedforward neural network, and the method includes:

acquiring a training set, wherein the training set comprises a plurality of training samples and true value samples corresponding to each training sample, each training sample comprises a graph structure of a plurality of continuous historical moments, each true value sample corresponding to each training sample comprises a graph structure of a plurality of future moments adjacent to the plurality of continuous historical moments, each graph structure of each moment comprises target data information of each server in a plurality of servers at the moment, and the target data information at least comprises load information;

Respectively inputting each graph structure in each training sample into the first graph convolution neural network, extracting a first road network space characteristic of each graph structure in each training sample, respectively inputting each graph structure in each truth sample into the second graph convolution neural network, and extracting a second road network space characteristic of each graph structure in each truth sample;

the convolution calculation of each first path network space feature through the first feedforward neural network is carried out to obtain a first path network space feature after the convolution calculation, and the convolution calculation of each second path network space feature through the third feedforward neural network is carried out to obtain a second path network space feature after the convolution calculation;

adding corresponding time information to the first road network space characteristics calculated by convolution at each moment, and inputting the first road network space characteristics of each training sample after adding the time information into the first multi-head attention mechanism network to obtain first road network space-time characteristics of each training sample;

the first path network space-time characteristics of each training sample are subjected to convolution calculation of the second feedforward neural network, so that the first path network space-time characteristics of each training sample after convolution calculation are obtained;

Inputting the first road network space-time characteristics after convolution calculation and a plurality of second road network space-time characteristics in corresponding truth value samples after time information addition into the second multi-head attention mechanism network to obtain third road network space-time characteristics of a plurality of future moments corresponding to each training sample and second road network space-time characteristics of each truth value sample;

inputting the third path network space-time characteristics of a plurality of future moments corresponding to each training sample and the second path network space-time characteristics of each truth sample into the fourth feedforward neural network to obtain the third path network space-time characteristics after convolution calculation and the second path network space-time characteristics after convolution calculation;

according to the third path network space-time characteristic after convolution calculation and the second path network space-time characteristic after convolution calculation, calculating a current loss value, adjusting model parameters of the load prediction model when the current loss value does not meet a convergence condition, and continuously training the load prediction model after parameter adjustment until the current loss value meets the convergence condition, and obtaining a final load prediction model.

In one possible implementation manner, the method for acquiring the target data information of each server includes:

Detecting abnormal values of the original data information of each server;

when detecting that an abnormal value exists in the original data information, repairing the abnormal value;

and carrying out normalization or standardization operation on the original data information after the abnormal value is repaired to obtain the target data information.

In one possible implementation manner, the detecting the outlier of the original data information of each server includes:

and performing dimension reduction processing on the original data information based on a Principal Component Analysis (PCA) algorithm, and detecting an outlier of the dimension reduced original data information by utilizing a Local Outlier Factor (LOF).

In one possible embodiment, the repairing the outlier includes:

repairing the abnormal value by using a generated countermeasure network to obtain repaired first data, and repairing the abnormal value by using a support vector regression algorithm to obtain repaired second data;

and carrying out weighted calculation on the first data and the second data to obtain a final result after repairing the abnormal value.

In one possible implementation, the target data information further includes: traffic and/or network performance.

In a second aspect, an embodiment of the present application provides a load prediction method of a server, where the method includes:

acquiring target data information of each server in the cloud computing platform at a plurality of latest historical moments, wherein the target data information at least comprises load information;

generating, for each historical moment, a graph structure of the each historical moment based on the target data information of the respective server in the cloud computing platform;

and inputting the graph structures of the latest historical moments into a load prediction model, and predicting the load information of each server of the future moments, wherein the load prediction model is trained according to the method of any implementation mode of the first aspect.

In a third aspect, embodiments of the present application provide a training apparatus of a load prediction model, the load prediction model including an encoder including a first graph roll-up neural network, a first feedforward neural network, a first multi-head attention mechanism network, a second feedforward neural network, and a decoder including a second graph roll-up neural network, a third feedforward neural network, a second multi-head attention mechanism network, and a fourth feedforward neural network, the apparatus comprising:

The training set comprises a plurality of training samples and true value samples corresponding to each training sample, each training sample comprises a graph structure of a plurality of continuous historical moments, each true value sample corresponding to each training sample comprises a graph structure of a plurality of future moments adjacent to the plurality of continuous historical moments, each graph structure of each moment comprises target data information of each server in a plurality of servers at the moment, and the target data information at least comprises load information;

the first feature extraction unit is used for respectively inputting each graph structure in each training sample into the first graph convolution neural network and extracting the first road network space feature of each graph structure in each training sample;

the second feature extraction unit is used for respectively inputting each graph structure in each truth value sample into the second graph convolution neural network and extracting a second road network space feature of each graph structure in each truth value sample;

the first convolution calculation unit is used for carrying out convolution calculation on each first path network space feature through the first feedforward neural network to obtain a first path network space feature after convolution calculation;

The second convolution calculation unit is used for carrying out convolution calculation on each second road network spatial feature through the third feedforward neural network to obtain a second road network spatial feature after convolution calculation;

the adding unit is used for adding corresponding time information to the first road network space characteristics after convolution calculation at each moment;

the first attention processing unit is used for inputting the first road network space characteristics of each training sample after time information is added into the first multi-head attention mechanism network to obtain the first road network space-time characteristics of each training sample;

the third convolution calculation unit is used for carrying out convolution calculation on the first path network space-time characteristics of each training sample through the second feedforward neural network to obtain first path network space-time characteristics of each training sample after convolution calculation;

the second attention processing unit is used for inputting the first path network space-time characteristics after convolution calculation and a plurality of second path network space-time characteristics in the corresponding truth value samples after time information addition into the second multi-head attention mechanism network to obtain third path network space-time characteristics of a plurality of future moments corresponding to each training sample and second path network space-time characteristics of each truth value sample;

A fourth convolution calculation unit, configured to input a third path network space-time feature of a future multiple moments corresponding to each training sample and a second path network space-time feature of each truth sample into the fourth feedforward neural network, to obtain a third path network space-time feature after convolution calculation and a second path network space-time feature after convolution calculation;

the adjustment training unit is used for calculating a current loss value according to the third path network space-time characteristic after convolution calculation and the second path network space-time characteristic after convolution calculation, adjusting the model parameters of the load prediction model when the current loss value does not meet the convergence condition, and continuing to train the load prediction model after parameter adjustment until the current loss value meets the convergence condition, so as to obtain a final load prediction model.

In one possible embodiment, the acquisition unit includes:

the abnormal value detection module is used for detecting abnormal values of the original data information of each server;

the restoration module is used for restoring the abnormal value when the abnormal value exists in the original data information;

and the processing module is used for carrying out normalization or standardization operation on the original data information after the abnormal value is repaired to obtain the target data information.

In one possible implementation manner, the outlier detection module is configured to perform a dimension reduction process on the raw data information based on a principal component analysis PCA algorithm, and perform outlier detection on the dimension reduced raw data information by using a local outlier factor LOF.

In a possible implementation manner, the repairing module is configured to repair the abnormal value by using a generating countermeasure network to obtain repaired first data, and repair the abnormal value by using a support vector regression algorithm to obtain repaired second data; and carrying out weighted calculation on the first data and the second data to obtain a final result after repairing the abnormal value.

In one possible implementation, the target data information further includes: traffic and/or network performance.

In a fourth aspect, an embodiment of the present application provides a load prediction apparatus of a server, where the apparatus includes:

the cloud computing platform comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring target data information of each server in the cloud computing platform at a plurality of latest historical moments, and the target data information at least comprises load information;

a generation unit configured to generate, for each history time, a graph structure of each history time based on the target data information of each server in the cloud computing platform;

And the prediction unit is used for inputting the graph structures of the latest historical moments into a load prediction model, and predicting the load information of each server of the future moments, wherein the load prediction model is trained according to the method of any implementation mode of the first aspect.

In a fifth aspect, embodiments of the present application provide a computer readable storage medium, on which a computer program is stored, which when executed by a processor implements a method as described in any one of the possible implementations of the first aspect, and implements a method as described in any one of the possible implementations of the second aspect.

In a sixth aspect, an embodiment of the present application provides an electronic device, including:

one or more processors;

the processor is coupled with a storage device for storing one or more programs;

when executed by one or more processors, causes an electronic device to implement a method as described in any one of the possible implementations of the first aspect, and implement a method as described in any one of the possible implementations of the second aspect.

In a seventh aspect, embodiments of the present application provide a computer program product, which contains instructions that, when executed on a computer or a processor, cause the computer or the processor to perform the method according to any one of the possible implementation manners of the first aspect, and implement the method according to any one of the possible implementation manners of the second aspect.

According to the load prediction method and device of the load prediction model, the load prediction model comprises an encoder and a decoder, the encoder and the decoder comprise a graph convolution neural network, a feedforward neural network and a multi-head attention mechanism network, each graph structure in each training sample sequentially passes through the first graph convolution neural network, the first feedforward neural network, the first multi-head attention mechanism network and the second feedforward neural network in the encoder, finally, the first path network space-time characteristic after the convolution calculation of each training sample is output, each graph structure in each training sample is sequentially input into the second graph convolution neural network and the third feedforward neural network, the second path network space-time characteristic after the convolution calculation is obtained, the third path network space-time characteristic of each training sample at a plurality of moments in the future is input into the fourth feedforward neural network, the third path network space-time characteristic after the convolution calculation and the second path network space-time characteristic after the convolution calculation are obtained, the load loss is not met according to the third path network space-time characteristic after the convolution calculation of each truth sample, the load loss is adjusted according to the load loss prediction model, and the load loss is not met, and the load prediction model is adjusted continuously. Therefore, the load information of each server in the cloud computing platform can be analyzed from the spatial and time levels, and long-term dependency relations can be captured, so that the load information of the servers can be predicted more accurately by the load prediction model obtained through training.

The innovations of the embodiments of the present application include at least:

1. the load prediction model provided by the embodiment of the application comprises an encoder and a decoder, wherein the encoder and the decoder comprise a graph convolution neural network (Graph convolution Network, GCN), a feedforward neural network and a multi-head attention mechanism network, the target data information is firstly converted into a graph structure comprising a server topological structure, then road network space characteristics are extracted from space dimensions based on the graph convolution neural network, the dependence relationship of each road network space characteristic is analyzed from the time dimensions based on the multi-head attention mechanism network, road network space-time characteristics are extracted from the road network space-time characteristics, and after each characteristic extraction, the extracted characteristics are refined through the feedforward neural network, so that the convergence of the load prediction model can be accelerated and the quality of the load prediction model can be improved. Therefore, the load prediction model trained by the embodiment of the application can analyze the load information of each server in the cloud computing platform from the spatial and time levels, and can capture long-term dependency, so that the load prediction model obtained by training can more accurately predict the load information of the server.

2. According to the method and the device, the abnormal value detection, repair and normalization (normalization) operation is carried out on the original data information, so that the target data information is more accurate, the model can be better adapted to different data distribution and change, and the performance and the robustness of the model are improved.

3. By combining the PCA (Principal Components Analysis, principal component analysis) algorithm with the LOF (Local Outlier Factor ) algorithm, the embodiment of the application can detect outliers more accurately and improve the robustness.

4. According to the embodiment of the application, the abnormal value is repaired by weighting based on the abnormal value repairing method of the GAN (Generative Adversarial Nets) and the abnormal value repairing method of the support vector regression, so that the accuracy of abnormal value repairing can be improved.

5. In the embodiment of the application, when the model is trained or predicted, the model input comprises the load information and also comprises the flow and/or the network performance, and the future load information is predicted by combining the historical flow and/or the network performance, so that the predicted result is more objective and is closer to the future true value, and the accuracy of the load prediction can be further improved.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following description will briefly introduce the drawings that are required to be used in the description of the embodiments or the prior art. It is apparent that the drawings in the following description are only some of the embodiments of the present application. Other figures may be derived from these figures without inventive effort for a person of ordinary skill in the art.

Fig. 1 is a flow chart of a training method of a load prediction model according to an embodiment of the present application;

fig. 2 is a schematic structural diagram of a load prediction model according to an embodiment of the present application;

fig. 3 is a flow chart of a load prediction method of a server according to an embodiment of the present application;

FIG. 4 is a block diagram of a training device for a load prediction model according to an embodiment of the present application;

fig. 5 is a block diagram of a load prediction device of a server according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. It will be apparent that the described embodiments are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without undue burden, are within the scope of the present application.

It should be noted that, in the case of no conflict, the embodiments and features in the embodiments may be combined with each other. The terms "comprising" and "having" and any variations thereof in the embodiments and figures of the present application are intended to cover non-exclusive inclusions. A process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed but may alternatively include other steps or elements not listed or inherent to such process, method, article, or apparatus.

Fig. 1 is a flow chart of a method for training a load prediction model, which can be applied to an electronic device or a computer device, and the method includes the following steps S110-S180. As shown in fig. 2, the load prediction model includes an encoder 210 including a first graph roll-up neural network 211, a first feed-forward neural network 212, a first multi-head attention mechanism network 213, a second feed-forward neural network 214, and a decoder 220 including a second graph roll-up neural network 221, a third feed-forward neural network 222, a second multi-head attention mechanism network 223, and a fourth feed-forward neural network 224.

The model training method is described in detail below:

s110: a training set is obtained.

The training set comprises a plurality of training samples and true value samples corresponding to each training sample, each training sample comprises a graph structure of a plurality of continuous historical moments, each true value sample corresponding to each training sample comprises a graph structure of a plurality of adjacent future moments of the plurality of continuous historical moments, each graph structure of each moment comprises target data information of each server in a plurality of servers at the moment, and the target data information at least comprises load information. The graph structure at each moment can be a topology structure of a plurality of servers, each node in the graph structure represents one server, and each node carries target data information.

The duration of the continuous plurality of historical time points may be the same as or different from the future plurality of time points adjacent to the continuous plurality of historical time points, and when the durations are different, the duration of the continuous plurality of historical time points is often longer than the duration of the future plurality of time points. For example, the continuous plurality of historical time points are 1 st to 5 th time points, and the future plurality of time points may be 6 to 10 time points or 6 to 8 time points.

The target data information may include, in addition to the load information: traffic and/or network performance. Traffic includes ingress traffic, egress traffic, total traffic, etc., and load information may include CPU (Central Processing Unit ) usage, memory usage, etc. Network performance includes latency, bandwidth utilization, etc.

In order to improve stability, generalization performance and robustness of the load prediction model, after the original data information (including at least the original load information) of each server is acquired, outlier detection may be performed on the original data information of each server. When the abnormal value exists in the original data information, repairing the abnormal value, and finally normalizing or standardizing the original data information after repairing the abnormal value to obtain the target data information.

The method for detecting the abnormal value of the original data information of each server comprises the following steps: and performing dimension reduction processing on the original data information based on a PCA algorithm, and performing outlier detection on the dimension reduced original data information by using LOF, namely calculating and identifying outlier factor scores of the dimension reduced original data information by using LOF, and determining outliers according to the scores. By combining the two methods of PCA and LOF, outliers can be detected more accurately and robustness is improved.

The method for repairing the abnormal value comprises the following steps: repairing the abnormal value by using the generated countermeasure network to obtain repaired first data, and repairing the abnormal value by using a support vector regression algorithm to obtain repaired second data; and carrying out weighted calculation on the first data and the second data to obtain a final result after the abnormal value is repaired.

Wherein a GAN model can be trained first, taking as input the raw data information, with the goal of generating synthetic data that is similar to the raw data information. After training is completed, the generator part is used for generating repaired abnormal value data; and simultaneously, the support vector regression is used for generating the repaired abnormal value data. And finally, carrying out weighted summation on the repaired data obtained in the two methods, and generating a repaired final result according to the weighted result.

According to the embodiment of the application, the abnormal values are detected and repaired, normalization or standardization and other operations are carried out on the data, so that the model can be better adapted to different data distribution and change, and the performance and the robustness of the model are improved.

After preprocessing the original data information to obtain target data information, determining the length of each piece of data according to the number of servers to be predicted, and then adjusting the data size, wherein the data to be input refers to all information at one moment, including traffic, load, network performance and the like, and each piece of data is in a graph structure. Ensuring that the tensor sizes of the input and output of the encoder and decoder in the load prediction model are consistent. These data sets may then be divided into training and testing sets. Since the embodiment of the application is time series data, the division of the training set is slightly different from the division of the general data set, a part of data is required to be used as historical data, and a part of data in the future of the historical data is required to be used as target data. The training set refers to the data used for learning by the load prediction model, and the test set refers to the data used for testing the performance of the model after training. The performance indexes of the model comprise accuracy, recall rate, mean square error and the like. Accuracy is expressed as the proportion of samples correctly predicted, and recall is expressed as the proportion of samples correctly predicted as positive among all the actual positive examples. The mean square error then represents the mean of the squared difference between the predicted value and the true value.

In addition, to enhance model input, position codes are computed before entering the training set into the load prediction model training, which treat each time step (each instant) as an absolute position, and use trigonometric functions (e.g., sine and cosine functions) to generate periodic codes of different frequencies. The local timestamp is a position code, and there may be a problem of error matching between the encoder and the decoder, so that the global timestamp is added as a time code in addition to the position code, and the influence of date factors such as week, month, holiday and the like can be considered by using the code, so that the accuracy of server load prediction can be improved.

S120: and respectively inputting each graph structure in each training sample into a first graph convolution neural network, extracting the first road network spatial characteristic of each graph structure in each training sample, respectively inputting each graph structure in each truth sample into a second graph convolution neural network, and extracting the second road network spatial characteristic of each graph structure in each truth sample.

In order to enable the load prediction model to extract spatial dependence, the embodiment of the application adds a graph convolution neural network in the encoder and the decoder, and performs convolution operation on the graph structure by using the graph convolution neural network. The convolution layer filters the input data through a set of learnable filters and extracts features. The core idea of the convolution layer is weight sharing, and all filters calculate the same convolution on the input data, so that parameters needing training can be reduced. The convolution operation can be regarded as a special linear transformation which converts local relations in the input data into global information and has translational invariance, i.e. the result of feature extraction is independent of the position of the input data, which makes the convolution layer very suitable for processing spatially and temporally adjacent data, can capture their inherent rules and structures and improves the robustness of the model.

S130: and carrying out convolution calculation on each first path network space feature through a first feedforward neural network to obtain first path network space features after convolution calculation, and carrying out convolution calculation on each second path network space feature through a third feedforward neural network to obtain second path network space features after convolution calculation.

The feedforward neural network is an artificial neural network, and the structure of the feedforward neural network is composed of a plurality of layers of nodes and transmits information according to a specific direction. The first path network space characteristic and the second path network space characteristic are respectively input into the feedforward neural network, so that the first path network space characteristic and the second path network space characteristic can be more refined.

S140: corresponding time information is added to the first road network space characteristics after convolution calculation at each moment, and the first road network space characteristics of each training sample after the time information is added are input into a first multi-head attention mechanism network to obtain first road network space-time characteristics of each training sample.

The time information corresponding to the convolutionally calculated first road network space characteristics at each moment is the moment at which the convolutionally calculated first road network space characteristics are located. After the first road network space characteristics calculated by convolution at each moment in each training sample are added with corresponding time information, all the first road network space characteristics in one training sample can be input into a first multi-head attention mechanism network so as to ensure that the model can consider the time dynamics of the road network characteristics and obtain the first road network moment characteristics of each training sample.

S150: and carrying out convolution calculation on the first path network space-time characteristics of each training sample through a second feedforward neural network to obtain the first path network space-time characteristics of each training sample after convolution calculation.

In order to make the first road network space-time characteristics more refined, the first road network space-time characteristics of each training sample can be subjected to convolution calculation of a second feedforward neural network to obtain the first road network space-time characteristics of each training sample after the convolution calculation.

S160: and inputting the first path network space-time characteristics after convolution calculation and a plurality of second path network space-time characteristics in the corresponding truth value samples after time information addition into a second multi-head attention mechanism network to obtain third path network space-time characteristics of a plurality of future moments corresponding to each training sample and the second path network space-time characteristics of each truth value sample.

After the encoder outputs the first path network space-time characteristics after convolution calculation, the first path network space-time characteristics can be input into a second multi-head attention mechanism network of a decoder to perform decoding calculation, a third path network space-time characteristic of a plurality of future moments corresponding to each training sample is obtained, and meanwhile, after corresponding time information is added to the second path network space characteristics after convolution calculation of each moment, the second path network space-time characteristics of each truth value sample after the time information is added are input into the second multi-head attention mechanism network, and the second path network space-time characteristics of each truth value sample are obtained.

S170: and inputting the third path network space-time characteristics of a plurality of moments in the future corresponding to each training sample and the second path network space-time characteristics of each truth value sample into a fourth feedforward neural network to obtain the third path network space-time characteristics after convolution calculation and the second path network space-time characteristics after convolution calculation.

In order to make the third-path network space-time feature and the second-path network space-time feature more refined, the third-path network space-time feature and the second-path network space-time feature can be respectively input into a fourth feedforward neural network to obtain the third-path network space-time feature after convolution calculation and the second-path network space-time feature after convolution calculation.

S180: according to the third path network space-time characteristic after the convolution calculation and the second path network space-time characteristic after the convolution calculation, calculating a current loss value, adjusting model parameters of the load prediction model when the current loss value does not meet the convergence condition, and continuously training the load prediction model after parameter adjustment until the current loss value meets the convergence condition, and obtaining a final load prediction model.

After the third path network space-time feature after the convolution calculation and the second path network space-time feature after the convolution calculation are obtained, the current loss value can be calculated according to the difference between the third path network space-time feature after the convolution calculation and the second path network space-time feature after the convolution calculation, for example, the difference between the third path network space-time feature after the convolution calculation and the second path network space-time feature after the convolution calculation corresponding to each pair of training samples and truth samples is calculated respectively, and then the average value of the difference values is used as the current loss value.

And when the current loss value is greater than or equal to the loss threshold value, determining that the current loss value does not meet the convergence condition, adjusting the model parameters of the load prediction model at the moment, and then continuously executing the steps S110-S170 based on the load prediction model after the parameters are adjusted until the current loss value is determined to meet the convergence condition when the current loss value is less than the loss threshold value, and taking the load prediction model obtained by current training as a final load prediction model.

In one embodiment, the loss function of the model training process is accomplished using a cross entropy loss function. The cross entropy loss function may measure the probability distribution of each set of future time instant target data information and compare the predicted probability with the actual target data information. At the same time, to avoid overfitting, regularization terms, such as L2 regularization, are also typically added.

After a converged load prediction model is obtained, a test data set is input into the load prediction model, the output result of the model is visualized or analyzed, and the performance index of the model is calculated and evaluated.

It should be added that, in the embodiment of the present application, residual connection may be performed between each network of the encoder and the decoder, and layer normalization operation is performed after the residual connection, so as to alleviate the gradient vanishing problem caused by adding depth in the deep neural network. And Dropout operation is used, i.e. the output of neurons in part of the network is discarded, reducing the overfitting phenomenon.

The decoder, like the encoder portion, has one more multi-headed attention mechanism for interacting with the encoder output than the encoder. The Multi-Head Attention layer calculates the Attention distribution between each position and all positions using each position in the input sequence as a query (Q), a key (K) and a value (V), resulting in a weighted sum representing the context information of the position. And the full connection layer performs forward propagation on the context information to obtain the output of the layer. Specifically, the Multi-Head Attention is calculated as follows:

wherein d _k Is vector dimension, Q,K. V is a vector. The calculation formula shows that for each query Q, multi-Head Attention will weight sum all values V according to their similarity to all keys K.

The second multi-headed attentiveness-mechanism network in the decoder is similar to the first multi-headed attentiveness-mechanism network in the encoder, but the second multi-headed attentiveness-mechanism network in the decoder only considers the position before that position when calculating the attentiveness profile, thereby avoiding the problem of using future information in the decoder.

According to the training method of the load prediction model, the load prediction model comprises an encoder and a decoder, the encoder and the decoder comprise a graph convolution neural network, a feedforward neural network and a multi-head attention mechanism network, each graph structure in each training sample sequentially passes through the first graph convolution neural network, the first feedforward neural network, the first multi-head attention mechanism network and the second feedforward neural network in the encoder, finally the first path network space-time characteristic after convolution calculation of each training sample is output, each graph structure in each truth sample is sequentially input into the second graph convolution neural network and the third feedforward neural network, the second path network space-time characteristic after convolution calculation is obtained, the third path network space-time characteristic at a plurality of moments in the future corresponding to each training sample is input into the fourth feedforward neural network, the third path network space-time characteristic after convolution calculation and the second path network space-time characteristic after convolution calculation are obtained, the current load loss is not met when the load loss is not met by the load prediction model, and the load prediction model is adjusted based on the current load loss prediction model. Therefore, the load information of each server in the cloud computing platform can be analyzed from the spatial and time levels, and long-term dependency relations can be captured, so that the load information of the servers can be predicted more accurately by the load prediction model obtained through training.

Based on the above method embodiment, another embodiment of the present application provides a load prediction method of a server, as shown in fig. 3, where the method includes:

s310: and acquiring target data information of each server in the cloud computing platform at a plurality of latest historical moments.

The target data information includes at least load information and may also include traffic and/or network performance. The traffic includes an input traffic, an output traffic, a total traffic, etc., and the load information may include a CPU usage rate, a memory usage rate, etc. Network performance includes latency, bandwidth utilization, etc.

When it is necessary to predict load information of each server at a plurality of times in the future, target data information at a plurality of times of history, for example, target data information at a plurality of times included in the last 10 minutes, may be acquired first. The time may be determined according to actual requirements, for example, one time may refer to 1 minute or 5 minutes.

In order to improve stability, generalization performance and robustness of the load prediction model, after obtaining the original data information of each server at a plurality of recent historical moments, outlier detection may be performed on the original data information of each server. When the abnormal value exists in the original data information, repairing the abnormal value, and finally normalizing or standardizing the original data information after repairing the abnormal value to obtain the target data information.

The method for detecting the abnormal value of the original data information of each server comprises the following steps: and performing dimension reduction processing on the original data information based on a PCA algorithm, and performing outlier detection on the dimension reduced original data information by using LOF, namely calculating and identifying outlier factor scores of the dimension reduced original data information by using LOF, and determining outliers according to the scores. By combining the two methods of PCA and LOF, outliers can be detected more accurately and robustness is improved.

The method for repairing the abnormal value comprises the following steps: repairing the abnormal value by using the generated countermeasure network to obtain repaired first data, and repairing the abnormal value by using a support vector regression algorithm to obtain repaired second data; and carrying out weighted calculation on the first data and the second data to obtain a final result after the abnormal value is repaired.

Wherein a GAN model can be trained first, taking as input the raw data information, with the goal of generating synthetic data that is similar to the raw data information. After training is completed, the generator part is used for generating repaired abnormal value data; and simultaneously, the support vector regression is used for generating the repaired abnormal value data. And finally, carrying out weighted summation on the repaired data obtained in the two methods, and generating a repaired final result according to the weighted result.

S320: for each historical moment, generating a graph structure of each historical moment based on target data information of each server in the cloud computing platform.

After obtaining the target data information of each server in the cloud computing platform, a graph structure of each historical moment can be generated according to the topological structure of each server, so that each node in the graph structure represents one server, and the information carried by each node comprises the target data information of the server at the historical moment.

S330: and inputting the graph structure of the latest historical moments into a load prediction model to predict the load information of each server at a plurality of future moments.

The load prediction model is trained according to the training method of the load prediction model provided by any embodiment. By inputting the graph structure of the latest multiple historical moments into a pre-trained load prediction model, and through the processing of an encoder and a decoder in the load prediction model, the load information of each server at multiple moments in the future can be predicted finally.

According to the load prediction method of the server, target data information of each server in the cloud computing platform at the latest historical moments can be acquired first, then for each historical moment, a graph structure of each historical moment is generated based on the target data information of each server in the cloud computing platform, and finally the graph structure of the latest historical moments is input into a load prediction model to predict the load information of each server at a plurality of future moments. The load prediction model comprises an encoder and a decoder, wherein the encoder and the decoder comprise a graph convolution neural network, a feedforward neural network and a multi-head attention mechanism network, the target data information is firstly converted into a graph structure comprising a server topological structure, then road network space characteristics are extracted from space dimensions based on the graph convolution neural network, the dependence relationship of each road network space characteristic is analyzed from time dimensions based on the multi-head attention mechanism network, road network space-time characteristics are extracted from the road network space-time characteristics, and after each characteristic extraction, the extracted characteristics are refined through the feedforward neural network, so that the convergence of the load prediction model can be accelerated, and the quality of the load prediction model can be improved. Therefore, the load prediction model trained by the embodiment of the application can analyze the load information of each server in the cloud computing platform from the spatial and time levels, and can capture long-term dependency, so that the load prediction model obtained by training can more accurately predict the load information of the server.

Based on the above method embodiments, another embodiment of the present application provides a training apparatus of a load prediction model, where the load prediction model includes an encoder and a decoder, the encoder includes a first graph roll-up neural network, a first feedforward neural network, a first multi-head attention mechanism network, and a second feedforward neural network, and the decoder includes a second graph roll-up neural network, a third feedforward neural network, a second multi-head attention mechanism network, and a fourth feedforward neural network, as shown in fig. 4, where the apparatus includes:

an obtaining unit 410, configured to obtain a training set, where the training set includes a plurality of training samples and a truth value sample corresponding to each training sample, each training sample includes a graph structure of a plurality of continuous historical moments, each truth value sample corresponding to each training sample includes a graph structure of a plurality of future moments adjacent to the plurality of continuous historical moments, each graph structure of each moment includes target data information of each server in a plurality of servers at the moment, and the target data information includes at least load information;

a first feature extraction unit 420, configured to input each graph structure in each training sample into the first graph convolution neural network, and extract a first road network spatial feature of each graph structure in each training sample;

A second feature extraction unit 430, configured to input each graph structure in each truth sample into the second graph convolution neural network, and extract a second road network spatial feature of each graph structure in each truth sample;

a first convolution calculating unit 440, configured to perform convolution calculation on each of the first road network spatial features through the first feedforward neural network to obtain a first road network spatial feature after the convolution calculation;

a second convolution calculating unit 450, configured to perform convolution calculation on each of the second path network spatial features through the third feedforward neural network to obtain a second path network spatial feature after the convolution calculation;

an adding unit 460, configured to add corresponding time information to the first path network spatial feature after the convolution calculation at each moment;

a first attention processing unit 470, configured to input the first road network spatial feature of each training sample after adding time information into the first multi-head attention mechanism network, to obtain a first road network spatial feature of each training sample;

a third convolution calculating unit 480, configured to perform convolution calculation on the first path network space-time feature of each training sample through the second feedforward neural network, to obtain a first path network space-time feature after convolution calculation of each training sample;

A second attention processing unit 490, configured to input the first path network space-time feature after convolution calculation and a plurality of second path network space-time features in the corresponding truth value samples after adding time information into the second multi-head attention mechanism network, to obtain a third path network space-time feature corresponding to each training sample at a plurality of future times, and a second path network space-time feature of each truth value sample;

a fourth convolution calculating unit 4100, configured to input a third path network space-time feature of each future multiple moments corresponding to each training sample and a second path network space-time feature of each truth sample into the fourth feedforward neural network, to obtain a third path network space-time feature after convolution calculation and a second path network space-time feature after convolution calculation;

the adjustment training unit 4110 is configured to calculate a current loss value according to the third path network space-time characteristic after the convolution calculation and the second path network space-time characteristic after the convolution calculation, adjust a model parameter of the load prediction model when the current loss value does not meet a convergence condition, and continuously train the load prediction model after the parameter adjustment until the current loss value meets the convergence condition, thereby obtaining a final load prediction model.

In one possible implementation, the acquisition unit 410 includes:

the abnormal value detection module is used for detecting abnormal values of the original data information of each server;

the restoration module is used for restoring the abnormal value when the abnormal value exists in the original data information;

and the processing module is used for carrying out normalization or standardization operation on the original data information after the abnormal value is repaired to obtain the target data information.

In one possible implementation manner, the outlier detection module is configured to perform a dimension reduction process on the raw data information based on a principal component analysis PCA algorithm, and perform outlier detection on the dimension reduced raw data information by using a local outlier factor LOF.

In a possible implementation manner, the repairing module is configured to repair the abnormal value by using a generating countermeasure network to obtain repaired first data, and repair the abnormal value by using a support vector regression algorithm to obtain repaired second data; and carrying out weighted calculation on the first data and the second data to obtain a final result after repairing the abnormal value.

In one possible implementation, the target data information further includes: traffic and/or network performance.

According to the training device of the load prediction model, the load prediction model comprises an encoder and a decoder, the encoder and the decoder comprise a graph convolution neural network, a feedforward neural network and a multi-head attention mechanism network, each graph structure in each training sample sequentially passes through the first graph convolution neural network, the first feedforward neural network, the first multi-head attention mechanism network and the second feedforward neural network in the encoder, finally the first path network space-time characteristic after convolution calculation of each training sample is output, each graph structure in each truth sample is sequentially input into the second graph convolution neural network and the third feedforward neural network, the second path network space-time characteristic after convolution calculation is obtained, the third path network space-time characteristic at a plurality of moments in the future corresponding to each training sample is input into the fourth feedforward neural network, the third path network space-time characteristic after convolution calculation and the second path network space-time characteristic after convolution calculation are obtained, the current load loss is not met when the load loss is not met by the load prediction model, and the load prediction model is adjusted based on the current load loss prediction model. Therefore, the load information of each server in the cloud computing platform can be analyzed from the spatial and time levels, and long-term dependency relations can be captured, so that the load information of the servers can be predicted more accurately by the load prediction model obtained through training.

Based on the above method embodiment, another embodiment of the present application provides a load prediction apparatus of a server, as shown in fig. 5, where the apparatus includes:

an obtaining unit 510, configured to obtain target data information of each server in the cloud computing platform at a plurality of recent historical moments, where the target data information at least includes load information;

a generating unit 520, configured to generate, for each historical time, a graph structure of the each historical time based on the target data information of the respective servers in the cloud computing platform;

the prediction unit 530 is configured to input the graph structures of the most recent historical moments into a load prediction model, and predict load information of each server at a plurality of future moments, where the load prediction model is trained according to the training method of the load prediction model provided in any one of the embodiments.

According to the load prediction device of the server, target data information of each server in the cloud computing platform at the latest historical moments can be acquired first, then for each historical moment, a graph structure of each historical moment is generated based on the target data information of each server in the cloud computing platform, and finally the graph structure of the latest historical moments is input into a load prediction model to predict the load information of each server at a plurality of future moments. The load prediction model comprises an encoder and a decoder, wherein the encoder and the decoder comprise a graph convolution neural network, a feedforward neural network and a multi-head attention mechanism network, the target data information is firstly converted into a graph structure comprising a server topological structure, then road network space characteristics are extracted from space dimensions based on the graph convolution neural network, the dependence relationship of each road network space characteristic is analyzed from time dimensions based on the multi-head attention mechanism network, road network space-time characteristics are extracted from the road network space-time characteristics, and after each characteristic extraction, the extracted characteristics are refined through the feedforward neural network, so that the convergence of the load prediction model can be accelerated, and the quality of the load prediction model can be improved. Therefore, the load prediction model trained by the embodiment of the application can analyze the load information of each server in the cloud computing platform from the spatial and time levels, and can capture long-term dependency, so that the load prediction model obtained by training can more accurately predict the load information of the server.

Based on the above method embodiments, another embodiment of the present application provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a method as described in any of the above embodiments.

Based on the above method embodiments, another embodiment of the present application provides an electronic device or a computer device, including:

one or more processors;

the processor is coupled with a storage device for storing one or more programs;

wherein the one or more programs, when executed by the one or more processors, cause the electronic or computer device to implement the methods of any of the embodiments described above.

Based on the above embodiments, another embodiment of the present application provides a computer program product, where instructions are included in the computer program product, which when executed on a computer or a processor, cause the computer or the processor to perform a method according to any of the above embodiments.

The device embodiment corresponds to the method embodiment, and has the same technical effects as the method embodiment, and the specific description refers to the method embodiment. The apparatus embodiments are based on the method embodiments, and specific descriptions may be referred to in the method embodiment section, which is not repeated herein. Those of ordinary skill in the art will appreciate that: the figures are schematic representations of one embodiment only and the modules or flows in the figures are not necessarily required to practice the present application.

Those of ordinary skill in the art will appreciate that: the modules in the apparatus of the embodiments may be distributed in the apparatus of the embodiments according to the description of the embodiments, or may be located in one or more apparatuses different from the present embodiments with corresponding changes. The modules of the above embodiments may be combined into one module, or may be further split into a plurality of sub-modules.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and are not limiting thereof; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the corresponding technical solutions.

Claims (10)

1. A method of training a load prediction model, the load prediction model comprising an encoder and a decoder, the encoder comprising a first graph roll-up neural network, a first feed-forward neural network, a first multi-head attention mechanism network, a second feed-forward neural network, the decoder comprising a second graph roll-up neural network, a third feed-forward neural network, a second multi-head attention mechanism network, a fourth feed-forward neural network, the method comprising:

Acquiring a training set, wherein the training set comprises a plurality of training samples and true value samples corresponding to each training sample, each training sample comprises a graph structure of a plurality of continuous historical moments, each true value sample corresponding to each training sample comprises a graph structure of a plurality of future moments adjacent to the plurality of continuous historical moments, each graph structure of each moment comprises target data information of each server in a plurality of servers at the moment, and the target data information at least comprises load information;

respectively inputting each graph structure in each training sample into the first graph convolution neural network, extracting a first road network space characteristic of each graph structure in each training sample, respectively inputting each graph structure in each truth sample into the second graph convolution neural network, and extracting a second road network space characteristic of each graph structure in each truth sample;

the convolution calculation of each first path network space feature through the first feedforward neural network is carried out to obtain a first path network space feature after the convolution calculation, and the convolution calculation of each second path network space feature through the third feedforward neural network is carried out to obtain a second path network space feature after the convolution calculation;

Adding corresponding time information to the first road network space characteristics calculated by convolution at each moment, and inputting the first road network space characteristics of each training sample after adding the time information into the first multi-head attention mechanism network to obtain first road network space-time characteristics of each training sample;

the first path network space-time characteristics of each training sample are subjected to convolution calculation of the second feedforward neural network, so that the first path network space-time characteristics of each training sample after convolution calculation are obtained;

inputting the first road network space-time characteristics after convolution calculation and a plurality of second road network space-time characteristics in corresponding truth value samples after time information addition into the second multi-head attention mechanism network to obtain third road network space-time characteristics of a plurality of future moments corresponding to each training sample and second road network space-time characteristics of each truth value sample;

inputting the third path network space-time characteristics of a plurality of future moments corresponding to each training sample and the second path network space-time characteristics of each truth sample into the fourth feedforward neural network to obtain the third path network space-time characteristics after convolution calculation and the second path network space-time characteristics after convolution calculation;

According to the third path network space-time characteristic after convolution calculation and the second path network space-time characteristic after convolution calculation, calculating a current loss value, adjusting model parameters of the load prediction model when the current loss value does not meet a convergence condition, and continuously training the load prediction model after parameter adjustment until the current loss value meets the convergence condition, and obtaining a final load prediction model.

2. The method according to claim 1, wherein the method for acquiring the target data information of each of the servers comprises:

detecting abnormal values of the original data information of each server;

when detecting that an abnormal value exists in the original data information, repairing the abnormal value;

and carrying out normalization or standardization operation on the original data information after the abnormal value is repaired to obtain the target data information.

3. The method of claim 2, wherein said anomaly value detection of raw data information for each of said servers comprises:

and performing dimension reduction processing on the original data information based on a Principal Component Analysis (PCA) algorithm, and detecting an outlier of the dimension reduced original data information by utilizing a Local Outlier Factor (LOF).

4. The method of claim 2, wherein the repairing the outlier comprises:

repairing the abnormal value by using a generated countermeasure network to obtain repaired first data, and repairing the abnormal value by using a support vector regression algorithm to obtain repaired second data;

and carrying out weighted calculation on the first data and the second data to obtain a final result after repairing the abnormal value.

5. The method of any of claims 1-4, wherein the target data information further comprises: traffic and/or network performance.

6. A method for predicting load of a server, the method comprising:

acquiring target data information of each server in the cloud computing platform at a plurality of latest historical moments, wherein the target data information at least comprises load information;

generating, for each historical moment, a graph structure of the each historical moment based on the target data information of the respective server in the cloud computing platform;

the graph structure of the latest historical moments is input into a load prediction model, and the load information of each server of the future moments is predicted, wherein the load prediction model is trained according to the method of any one of claims 1-5.

7. A training apparatus for a load prediction model, the load prediction model comprising an encoder and a decoder, the encoder comprising a first graph roll-up neural network, a first feed-forward neural network, a first multi-head attention mechanism network, a second feed-forward neural network, the decoder comprising a second graph roll-up neural network, a third feed-forward neural network, a second multi-head attention mechanism network, a fourth feed-forward neural network, the apparatus comprising:

the training set comprises a plurality of training samples and true value samples corresponding to each training sample, each training sample comprises a graph structure of a plurality of continuous historical moments, each true value sample corresponding to each training sample comprises a graph structure of a plurality of future moments adjacent to the plurality of continuous historical moments, each graph structure of each moment comprises target data information of each server in a plurality of servers at the moment, and the target data information at least comprises load information;

the first feature extraction unit is used for respectively inputting each graph structure in each training sample into the first graph convolution neural network and extracting the first road network space feature of each graph structure in each training sample;

The second feature extraction unit is used for respectively inputting each graph structure in each truth value sample into the second graph convolution neural network and extracting a second road network space feature of each graph structure in each truth value sample;

the first convolution calculation unit is used for carrying out convolution calculation on each first path network space feature through the first feedforward neural network to obtain a first path network space feature after convolution calculation;

the second convolution calculation unit is used for carrying out convolution calculation on each second road network spatial feature through the third feedforward neural network to obtain a second road network spatial feature after convolution calculation;

the adding unit is used for adding corresponding time information to the first road network space characteristics after convolution calculation at each moment;

the first attention processing unit is used for inputting the first road network space characteristics of each training sample after time information is added into the first multi-head attention mechanism network to obtain the first road network space-time characteristics of each training sample;

the third convolution calculation unit is used for carrying out convolution calculation on the first path network space-time characteristics of each training sample through the second feedforward neural network to obtain first path network space-time characteristics of each training sample after convolution calculation;

The second attention processing unit is used for inputting the first path network space-time characteristics after convolution calculation and a plurality of second path network space-time characteristics in the corresponding truth value samples after time information addition into the second multi-head attention mechanism network to obtain third path network space-time characteristics of a plurality of future moments corresponding to each training sample and second path network space-time characteristics of each truth value sample;

a fourth convolution calculation unit, configured to input a third path network space-time feature of a future multiple moments corresponding to each training sample and a second path network space-time feature of each truth sample into the fourth feedforward neural network, to obtain a third path network space-time feature after convolution calculation and a second path network space-time feature after convolution calculation;

the adjustment training unit is used for calculating a current loss value according to the third path network space-time characteristic after convolution calculation and the second path network space-time characteristic after convolution calculation, adjusting the model parameters of the load prediction model when the current loss value does not meet the convergence condition, and continuing to train the load prediction model after parameter adjustment until the current loss value meets the convergence condition, so as to obtain a final load prediction model.

8. A load predicting apparatus of a server, the apparatus comprising:

the cloud computing platform comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring target data information of each server in the cloud computing platform at a plurality of latest historical moments, and the target data information at least comprises load information;

a generation unit configured to generate, for each history time, a graph structure of each history time based on the target data information of each server in the cloud computing platform;

the prediction unit is configured to input the graph structures of the most recent historical moments into a load prediction model, and predict load information of each server of the plurality of future moments, where the load prediction model is trained according to the method of any one of claims 1-5.

9. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any one of claims 1-5 or the method according to claim 6.

10. An electronic device, the electronic device comprising:

one or more processors;

the processor is coupled with a storage device for storing one or more programs;

The one or more programs, when executed by the one or more processors, cause the electronic device to perform the method of any of claims 1-5 or the method of claim 6.