CN117786566A - Load prediction model training method, server load prediction method and device - Google Patents

Abstract

This application discloses a load prediction model training method, a server load prediction method and a device. The model includes an encoder and a decoder. The encoder includes a first graph convolutional neural network, a first feedforward neural network, a first multi-layer neural network, and a first graph convolutional neural network. The head attention mechanism network, the second feedforward neural network, the decoder includes the second graph convolutional neural network, the third feedforward neural network, the second multi-head attention mechanism network, and the fourth feedforward neural network. The method includes: Each graph structure in each training sample is input into each network in the encoder in turn, and the spatiotemporal characteristics of the first network after convolution calculation of each training sample are obtained. Each graph structure in each true value sample is input into the second network in turn. The graph convolutional neural network and the third feedforward neural network obtain the spatial characteristics of the second network after convolution calculation, and input the spatiotemporal characteristics of the first network and the spatial characteristics of the second network into the second multi-head attention mechanism network in sequence. After the fourth feedforward neural network, the final model is obtained by adjusting the training.

Description

Load prediction model training method, server load prediction method and device

Technical Field

The present application relates to the field of cloud computing technology, specifically, to a load prediction model training method, a server load prediction method and a device.

Background technique

Cloud computing platform is one of the important development directions in the current information technology field. By providing elastic resources and flexible service models, it meets the needs of enterprises and individuals for efficient, scalable and reliable computing capabilities. In cloud computing platforms, server load prediction is a key task, which can help cloud service providers plan and manage resources effectively to ensure user service quality, while also reducing costs and increasing profits.

In related technologies, many load forecasting models based on statistical methods have been proposed and applied, such as time series models (such as ARIMA (Autoregressive Integrated Moving Average) model, SARIMA (Seasonal Autoregressive Integrated Moving Average) model), regression models (such as linear regression, support vector machine, etc.) and some deep learning models (such as recurrent neural network, long short-term memory network and convolutional neural network). However, these traditional methods are usually difficult to handle complex nonlinear relationships and spatiotemporal dependencies, so there are certain limitations in prediction accuracy.

Contents of the invention

This application provides a load prediction model training method, a server load prediction method and a device, which can improve the accuracy of load prediction.

The specific technical solutions are as follows:

In a first aspect, embodiments of the present application provide a method for training a load prediction model. The load prediction model includes an encoder and a decoder. The encoder includes a first graph convolutional neural network and a first feedforward neural network. , the first multi-head attention mechanism network, the second feed-forward neural network, the decoder includes a second graph convolutional neural network, a third feed-forward neural network, a second multi-head attention mechanism network, a fourth feed-forward neural network Neural network, the method includes:

Obtain a training set, wherein the training set includes a plurality of training samples and a true value sample corresponding to each training sample, each training sample includes a graph structure of multiple consecutive historical moments, and each training sample The corresponding true value sample includes the graph structure of multiple consecutive future moments adjacent to the multiple historical moments. The graph structure of each moment includes the target data information of each of the multiple servers at that moment, so The target data information at least includes load information;

input each graph structure in each training sample into the first graph convolutional neural network, extract the first network space feature of each graph structure in each training sample, and Each graph structure in each true value sample is input into the second graph convolutional neural network, and the second network space feature of each graph structure in each true value sample is extracted. ;

Each of the first road network spatial features is subjected to the convolution calculation of the first feedforward neural network to obtain the first road network spatial features after the convolution calculation, and each of the second road network spatial features is obtained After the convolution calculation of the third feedforward neural network, the second path network spatial characteristics after the convolution calculation are obtained;

The corresponding time information is added to the first path network spatial feature after the convolution calculation at each moment, and the first path network spatial feature of each training sample after adding the time information is input into the first multi-head attention In the force mechanism network, obtain the first network spatiotemporal characteristics of each training sample;

Pass the first road network spatio-temporal characteristics of each training sample through the convolution calculation of the second feedforward neural network to obtain the first road network spatio-temporal characteristics after the convolution calculation of each training sample;

Input the first road network spatiotemporal features after convolution calculation and the multiple second road network spatial features in the corresponding true value samples after adding time information into the second multi-head attention mechanism network to obtain the third road network spatiotemporal features at multiple future moments corresponding to each of the training samples and the second road network spatiotemporal features of each of the true value samples;

Input the spatio-temporal characteristics of the third network at multiple times in the future corresponding to each training sample and the spatio-temporal characteristics of the second network of each true value sample into the fourth feedforward neural network to obtain convolution calculations The spatio-temporal characteristics of the third road network after the convolution calculation and the spatio-temporal characteristics of the second road network after the convolution calculation;

According to the third road network spatiotemporal characteristics after convolution calculation and the second road network spatiotemporal characteristics after convolution calculation, the current loss value is calculated. When the current loss value does not meet the convergence condition, the model parameters of the load prediction model are adjusted, and the load prediction model after parameter adjustment is continued to be trained until the current loss value meets the convergence condition, so as to obtain the final load prediction model.

In a possible implementation, the method for obtaining target data information of each server includes:

Perform outlier detection on the original data information of each server;

When an outlier is detected in the original data information, repair the outlier;

Perform a normalization or standardization operation on the original data information after outlier repair to obtain the target data information.

In a possible implementation manner, performing outlier detection on the original data information of each server includes:

The original data information is subjected to dimensionality reduction processing based on the principal component analysis (PCA) algorithm, and outlier detection is performed on the original data information after dimensionality reduction using the local outlier factor (LOF).

In a possible implementation, repairing the outliers includes:

Using a generative adversarial network to repair the outliers to obtain the repaired first data, and using a support vector regression algorithm to repair the outliers to obtain the repaired second data;

A weighted calculation is performed on the first data and the second data to obtain a final result after the outlier is repaired.

In a possible implementation, the target data information also includes: traffic and/or network performance.

In a second aspect, embodiments of the present application provide a server load prediction method, which method includes:

Obtain target data information of each server in the cloud computing platform at multiple recent historical moments, where the target data information at least includes load information;

For each historical moment, generate the graph structure of each historical moment based on the target data information of each server in the cloud computing platform;

The graph structure of the most recent multiple historical moments is input into a load prediction model to predict the load information of each server at multiple future moments, wherein the load prediction model is trained according to the method described in any implementation method of the first aspect.

In a third aspect, embodiments of the present application provide a training device for a load prediction model. The load prediction model includes an encoder and a decoder. The encoder includes a first graph convolutional neural network and a first feedforward neural network. , the first multi-head attention mechanism network, the second feed-forward neural network, the decoder includes a second graph convolutional neural network, a third feed-forward neural network, a second multi-head attention mechanism network, a fourth feed-forward neural network Neural network, the device includes:

An acquisition unit is used to acquire a training set, wherein the training set includes a plurality of training samples and a true value sample corresponding to each training sample, and each training sample includes a graph structure of multiple consecutive historical moments. The true value samples corresponding to each of the training samples include the graph structure of multiple consecutive future moments adjacent to the multiple historical moments, and the graph structure of each moment includes multiple servers at that moment. Target data information, the target data information at least includes load information;

A first feature extraction unit, configured to input each graph structure in each training sample into the first graph convolutional neural network, and extract the characteristics of each graph structure in each training sample. The spatial characteristics of the first road network;

A second feature extraction unit, configured to input each graph structure in each true value sample into the second graph convolutional neural network, and extract each graph structure in each true value sample. The second network spatial characteristics of the structure;

A first convolution calculation unit, configured to perform convolution calculation on each of the first road network spatial features of the first feedforward neural network to obtain the first road network spatial features after convolution calculation;

A second convolution calculation unit is used to perform convolution calculation on each of the second road network spatial features of the third feedforward neural network to obtain the second road network spatial features after convolution calculation;

Add a unit to add corresponding time information to the spatial characteristics of the first network after the convolution calculation at each moment;

A first attention processing unit, configured to input the first road network spatial feature of each of the training samples after adding time information into the first multi-head attention mechanism network to obtain the first road network spatiotemporal feature of each of the training samples;

The third convolution calculation unit is used to pass the first channel spatiotemporal characteristics of each training sample through the convolution calculation of the second feedforward neural network to obtain the first channel network after the convolution calculation of each training sample. Network spatiotemporal characteristics;

The second attention processing unit is used to input the first road network spatiotemporal features after convolution calculation and the multiple second road network spatial features in the corresponding true value samples after adding time information into the second multi-head attention mechanism network to obtain the third road network spatiotemporal features at multiple future moments corresponding to each of the training samples and the second road network spatiotemporal features of each of the true value samples;

The fourth convolution calculation unit is used to input the spatio-temporal features of the third road network at multiple times in the future corresponding to each training sample, and the spatio-temporal features of the second road network for each of the true value samples into the fourth convolution calculation unit. The feedforward neural network obtains the spatio-temporal characteristics of the third network after convolution calculation and the spatio-temporal characteristics of the second network after convolution calculation;

Adjust the training unit to calculate the current loss value based on the spatio-temporal characteristics of the third road network after convolution calculation and the spatio-temporal characteristics of the second road network after convolution calculation. When the current loss value does not meet the convergence condition, all Adjust the model parameters of the load prediction model, and continue to train the load prediction model after parameter adjustment until the current loss value meets the convergence condition, and the final load prediction model is obtained.

In a possible implementation, the acquisition unit includes:

An outlier detection module, used for performing outlier detection on the original data information of each server;

A repair module, configured to repair an abnormal value when an abnormal value is detected in the original data information;

A processing module, configured to normalize or standardize the original data information after outlier repair to obtain the target data information.

In a possible implementation, the outlier detection module is used to perform dimensionality reduction processing on the original data information based on the principal component analysis PCA algorithm, and use the local outlier factor LOF to perform dimensionality reduction on the original data information after dimensionality reduction. Perform outlier detection.

In one possible implementation, the repair module is used to repair the outlier using a generative adversarial network to obtain repaired first data, and to repair the outlier using a support vector regression algorithm to obtain repaired second data; and to perform weighted calculation on the first data and the second data to obtain a final result after repairing the outlier.

In a possible implementation manner, the target data information further includes: traffic and/or network performance.

In the fourth aspect, embodiments of the present application provide a load prediction device for a server, where the device includes:

An acquisition unit, configured to acquire target data information of each server in the cloud computing platform at multiple recent historical moments, where the target data information at least includes load information;

A generation unit configured to generate, for each historical moment, the graph structure of each historical moment based on the target data information of each server in the cloud computing platform;

A prediction unit is used to input the graph structure of the most recent multiple historical moments into a load prediction model to predict the load information of each server at multiple future moments, wherein the load prediction model is trained according to the method described in any implementation method of the first aspect.

In a fifth aspect, embodiments of the present application provide a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method described in any possible implementation manner of the first aspect is implemented, and the method is implemented as follows: The method described in any possible implementation of the second aspect.

In a sixth aspect, an embodiment of the present application provides an electronic device, the electronic device comprising:

one or more processors;

The processor is coupled to a storage device for storing one or more programs;

When one or more programs are executed by one or more processors, the electronic device implements the method described in any possible implementation of the first aspect, and implements the method described in any possible implementation of the first aspect, Implement the method described in any possible implementation manner of the second aspect.

In a seventh aspect, embodiments of the present application provide a computer program product. The computer program product contains instructions. When the instructions are run on a computer or processor, the computer or processor causes the computer or processor to execute any possible method of the first aspect. The method described in the implementation manner implements the method described in any possible implementation manner of the second aspect.

Embodiments of the present application provide a load prediction model training method, a server load prediction method and a device. The load prediction model includes an encoder and a decoder, and both the encoder and the decoder include a graph convolutional neural network and a feedforward neural network. network, multi-head attention mechanism network, each graph structure in each training sample sequentially passes through the first graph convolutional neural network, the first feedforward neural network, the first multi-head attention mechanism network and the second feedforward neural network in the encoder. The feed neural network finally outputs the spatio-temporal characteristics of the first network after convolution calculation of each training sample, and inputs each graph structure in each true value sample into the second graph convolutional neural network and the third feedforward neural network in turn. Obtain the spatial features of the second road network after convolution calculation, input the spatio-temporal features of the third road network at multiple times in the future corresponding to each training sample, and input the spatio-temporal features of the second road network of each true value sample into the fourth front Feed the neural network to obtain the spatio-temporal characteristics of the third channel network after convolution calculation and the spatio-temporal characteristics of the second channel network after convolution calculation. According to the spatio-temporal characteristics of the third channel network after convolution calculation and the second channel network after convolution calculation, The spatio-temporal characteristics of the network are used to calculate the current loss value. When the current loss value does not meet the convergence conditions, the model parameters of the load prediction model are adjusted, and the load prediction model after parameter adjustment continues to be trained until the current loss value meets the convergence conditions. , obtain the final load prediction model in order to perform load prediction based on the final load prediction model. It can be seen from this that the embodiments of the present application can analyze the load information of each server in the cloud computing platform from the spatial and temporal levels, and can capture long-term dependencies, so that the trained load prediction model can more accurately predict the load information of the server. .

The innovativeness of the embodiments of this application at least includes:

1. The load prediction model provided by the embodiment of this application includes an encoder and a decoder, and both the encoder and the decoder include a graph convolution neural network (GCN), a feedforward neural network, and a multi-head attention mechanism network. , the embodiment of this application first converts the target data information into a graph structure including the server topology, then extracts the spatial characteristics of the road network from the spatial dimension based on the graph convolutional neural network, and then analyzes each road network from the time dimension based on the multi-head attention mechanism network. The spatial and temporal features of the road network are extracted from the dependencies of spatial features, and after each feature extraction, the extracted features are refined through a feedforward neural network, which can speed up the convergence of the load prediction model and improve the quality of the load prediction model. Therefore, the load prediction model trained in the embodiments of this application can analyze the load information of each server in the cloud computing platform from the spatial and temporal levels, and can capture long-term dependencies, so that the trained load prediction model can predict the server more accurately. load information.

2. The embodiments of the present application can make the target data information more accurate by performing outlier detection, repair, and normalization operations on the original data information, so that the model can better adapt to different data distributions and changes, thereby improving the performance and robustness of the model.

3. The embodiment of this application combines the PCA (Principal Components Analysis, principal component analysis) algorithm with the LOF (Local Outlier Factor, local outlier factor) algorithm to more accurately detect outliers and improve robustness.

4. The embodiment of this application performs weighted repair of outliers based on the outlier repair method of GAN (Generative Adversarial Nets, Generative Adversarial Nets) and the support vector regression outlier repair method, which can improve the accuracy of outlier repair.

5. During model training or model prediction in the embodiment of this application, in addition to load information, the model input also includes traffic and/or network performance. The future load information is predicted by combining historical traffic and/or network performance, so that the prediction The results are more objective and closer to the true value in the future, which can further improve the accuracy of load prediction.

Description of drawings

In order to more clearly explain the embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without exerting creative efforts.

Figure 1 is a schematic flow chart of a load prediction model training method provided by an embodiment of the present application;

Figure 2 is a schematic structural diagram of a load prediction model provided by an embodiment of the present application;

Figure 3 is a schematic flowchart of a server load prediction method provided by an embodiment of the present application;

Figure 4 is a block diagram of a training device for a load prediction model provided by an embodiment of the present application;

FIG. 5 is a block diagram of a server load prediction device provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of this application.

It should be noted that, as long as there is no conflict, the embodiments and features in the embodiments of this application can be combined with each other. The terms "including" and "having" and any variations thereof in the embodiments and drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes Other steps or units inherent to such processes, methods, products or devices.

Figure 1 is a schematic flowchart of a load prediction model training method. The method can be applied to electronic equipment or computer equipment. The method includes the following steps S110-S180. As shown in Figure 2, the load prediction model includes an encoder 210 and a decoder 220. The encoder includes a first graph convolutional neural network 211, a first feedforward neural network 212, a first multi-head attention mechanism network 213, a second Feedforward neural network 214, the decoder includes a second graph convolutional neural network 221, a third feedforward neural network 222, a second multi-head attention mechanism network 223, and a fourth feedforward neural network 224.

The following is a detailed introduction to the model training method:

S110: Obtain the training set.

The training set includes multiple training samples and true value samples corresponding to each training sample. Each training sample includes a graph structure of multiple consecutive historical moments. The true value sample corresponding to each training sample includes a graph structure of multiple future moments adjacent to multiple consecutive historical moments. The graph structure at each moment includes target data information of each server among multiple servers at that moment. The target data information includes at least load information. The graph structure at each moment can be a topological structure of multiple servers. Each node in the graph structure represents a server, and each node carries target data information.

The durations of multiple consecutive historical moments and multiple future moments adjacent to the multiple consecutive historical moments may be the same or different. When the durations are different, the duration of multiple consecutive historical moments is often greater than the duration of the multiple future moments. For example, multiple consecutive historical moments are moments 1-5, and multiple future moments can be moments 6-10, or moments 6-8.

In addition to load information, target data information may also include: traffic and/or network performance. Traffic includes incoming traffic, outgoing traffic, and total traffic. Load information can include CPU (Central Processing Unit, central processing unit) usage, memory usage, etc. Network performance includes latency, bandwidth utilization, etc.

In order to improve the stability, generalization performance and robustness of the load prediction model, after obtaining the original data information of each server (including at least the original load information), you can first perform outlier analysis on the original data information of each server. detection. When outliers are detected in the original data information, the outliers are repaired. Finally, the original data information after the outlier repair can be normalized or standardized to obtain the target data information.

Among them, the method for detecting outliers on the original data information of each server includes: performing dimensionality reduction processing on the original data information based on the PCA algorithm, and using LOF to detect outliers on the original data information after dimensionality reduction, that is, using LOF to calculate and identify the outlier factor score of the original data information after dimensionality reduction, and determining the outlier according to the score. By combining the two methods of PCA and LOF, outliers can be detected more accurately and robustness can be improved.

Methods for repairing outliers include: using a generative adversarial network to repair the outliers to obtain the repaired first data, and using the support vector regression algorithm to repair the outliers to obtain the repaired second data; The first data and the second data are weighted to obtain the final result after repairing the outliers.

Among them, a GAN model can be trained first, taking the original data information as input, and the goal is to generate synthetic data similar to the original data information. After training is completed, use the generator part to generate the repaired outlier data; at the same time, use support vector regression to generate the repaired outlier data. Finally, the repaired data obtained by these two methods are weighted and summed, and the final result after repair is generated based on the weighted results.

By detecting and repairing outliers and performing operations such as normalizing or standardizing the data, the embodiments of the present application can make the model better adapt to different data distributions and changes, thereby improving the performance and robustness of the model.

After preprocessing the original data information and obtaining the target data information, the length of each piece of data can be determined based on the number of servers that need to be predicted, and then the data size can be adjusted. The data that needs to be input here refers to all the information at one time. Including traffic, load, network performance, etc., and each piece of data is a graph structure. Ensure that the input and output tensor sizes of the encoder and decoder in the load prediction model are consistent. These data sets can then be divided into training and test sets. Since the embodiment of this application is time series data, the division of the training set is slightly different from the division of general data sets. It is necessary to use a part of the data as historical data and a part of the future data of the historical data as target data. The training set refers to the data used by the load prediction model to learn, and the test set refers to the data used to test the performance of the model after training. The performance indicators of the model include accuracy, recall and mean square error. The precision rate represents the proportion of correctly predicted samples, and the recall rate represents the proportion of correctly predicted positive samples among all actual positive samples. The mean square error represents the mean squared difference between the predicted value and the true value.

In addition, in order to enhance the model input, the position encoding is calculated before the training set is fed into the load prediction model training. The position encoding treats each time step (each moment) as an absolute position and uses trigonometric functions (such as sine and cosine functions) to generate periodic encodings of different frequencies. The local timestamp is used as the position encoding, which may cause the query and key to be mismatched between the encoder and decoder. Therefore, in addition to the position encoding, the global timestamp is also added as the time encoding. This encoding can take into account the influence of date factors such as weeks, months, holidays, etc., and improve the accuracy of server load prediction.

S120: Input each graph structure in each training sample into the first graph convolutional neural network, extract the first network spatial characteristics of each graph structure in each training sample, and convert each graph structure in each true value sample into the first graph convolutional neural network. Each graph structure is input into the second graph convolutional neural network respectively, and the second network spatial characteristics of each graph structure in each true value sample are extracted.

In order to enable the load prediction model to extract spatial dependence, embodiments of the present application add a graph convolutional neural network to the encoder and decoder, and use the graph convolutional neural network to perform convolution operations on the graph structure. The convolutional layer filters the input data and extracts features through a set of learnable filters. The core idea of the convolutional layer is weight sharing. All filters calculate the same convolution on the input data, thereby reducing the parameters that need to be trained. The convolution operation can be regarded as a special linear transformation, which converts local relationships in the input data into global information, and has translation invariance, that is, the result of feature extraction has nothing to do with the position of the input data, which makes the convolution layer It is very suitable for processing spatially and temporally adjacent data, capable of capturing their inherent patterns and structures, and improving the robustness of the model.

S130: Pass each first network spatial feature through the convolution calculation of the first feedforward neural network to obtain the first network spatial feature after the convolution calculation, and pass each second road network spatial feature through the third feedforward neural network. The convolution calculation of the fed neural network is performed to obtain the spatial characteristics of the second network after the convolution calculation.

Feedforward neural network is an artificial neural network whose structure consists of multiple levels of nodes and transmits information in a specific direction. Inputting the spatial features of the first road network and the spatial features of the second road network into the feedforward neural network respectively can make the spatial features of the first road network and the spatial features of the second road network more refined.

S140: Add corresponding time information to the first channel network spatial feature after convolution calculation at each moment, and input the first channel network spatial feature of each training sample after adding time information into the first multi-head attention mechanism In the network, the first network spatiotemporal characteristics of each training sample are obtained.

Among them, the time information corresponding to the first road network spatial feature calculated by convolution at each moment is the time at which the first road network spatial feature calculated by convolution is located. After adding the corresponding time information to the first path network spatial features calculated by convolution at each time in each training sample, all the first path network spatial features in a training sample can be input into the first multi-head attention mechanism network , to ensure that the model can consider the time dynamics of road network characteristics and obtain the first road network moment characteristics of each training sample.

S150: The spatio-temporal characteristics of the first network of each training sample are subjected to the convolution calculation of the second feedforward neural network to obtain the spatio-temporal characteristics of the first network after convolution calculation of each training sample.

In order to make the first road network spatiotemporal features more refined, the first road network spatiotemporal features of each training sample may be subjected to convolution calculation of the second feedforward neural network to obtain the first road network spatiotemporal features after convolution calculation of each training sample.

S160: Input the spatio-temporal features of the first path network calculated by convolution and the spatial features of the second path network in the corresponding true value samples after adding time information into the second multi-head attention mechanism network to obtain each The spatio-temporal characteristics of the third road network at multiple times in the future corresponding to the training sample, and the spatio-temporal characteristics of the second road network for each true value sample.

After the encoder outputs the spatio-temporal features of the first network after convolution calculation, it can be input into the second multi-head attention mechanism network of the decoder for decoding calculations to obtain the third network at multiple times in the future corresponding to each training sample. spatio-temporal features, and after adding corresponding time information to the second path network spatial feature after the convolution calculation at each moment, input the second path network spatial feature of each true value sample after adding the time information. In the head attention mechanism network, the second network spatiotemporal characteristics of each true value sample are obtained.

S170: Input the spatio-temporal characteristics of the third network at multiple times in the future corresponding to each training sample, and the spatio-temporal characteristics of the second network of each true value sample into the fourth feedforward neural network, and obtain the third network after convolution calculation. The spatio-temporal characteristics of the road network and the spatio-temporal characteristics of the second road network after convolution calculation.

In order to make the third road network spatiotemporal features and the second road network spatiotemporal features more refined, the third road network spatiotemporal features and the second road network spatiotemporal features can be respectively input into the fourth feedforward neural network to obtain the third road network spatiotemporal features after convolution calculation and the second road network spatiotemporal features after convolution calculation.

S180: Calculate the current loss value based on the spatiotemporal characteristics of the third road network after convolution calculation and the spatiotemporal characteristics of the second road network after convolution calculation. When the current loss value does not meet the convergence conditions, adjust the model parameters of the load prediction model, and continue to train the load prediction model after the parameter adjustment until the current loss value meets the convergence conditions, and obtain the final load prediction model.

After obtaining the spatio-temporal characteristics of the third road network after convolution calculation and the spatio-temporal characteristics of the second road network after convolution calculation, it can be based on the spatio-temporal characteristics of the third road network after convolution calculation and the second road network after convolution calculation. The difference between spatio-temporal features is used to calculate the current loss value. For example, for each pair of training samples and true value samples, the spatio-temporal features of the third path network after convolution calculation and the corresponding second path network after convolution calculation are calculated. The difference between spatio-temporal features, and then the average of these differences is used as the current loss value.

When the current loss value is greater than or equal to the loss threshold, it is determined that the current loss value does not meet the convergence condition. At this time, the model parameters of the load prediction model can be adjusted, and then based on the load prediction model after adjusting the parameters, steps S110-S170 are continued. Until the current loss value is less than the loss threshold, it is determined that the current loss value meets the convergence condition, and the load prediction model currently trained is used as the final load prediction model.

In one implementation, the loss function of the model training process is accomplished using a cross-entropy loss function. The cross-entropy loss function can measure the probability distribution of target data information at each set of future moments and compare the predicted probability with the real target data information. At the same time, in order to avoid overfitting, regularization terms are usually added, such as L2 regularization.

After obtaining a converged load forecasting model, the test data set is input into the load forecasting model, the output results of the model are visualized or analyzed, and its performance indicators are calculated and evaluated.

It should be added that the embodiments of the present application can perform residual connections between the networks of the encoder and the decoder, and perform layer normalization operations after the residual connections, which alleviates the problems caused by increasing depth in the deep neural network. The vanishing gradient problem. And the Dropout operation is used, that is, the output of some neurons in the network is discarded to reduce the over-fitting phenomenon.

The decoder is the same as the encoder part. Compared with the encoder, it has an additional multi-head attention mechanism for interacting with the encoder output. The Multi-Head Attention layer treats each position in the input sequence as a query (Q), key (K), and value (V), and calculates the attention distribution between each position and all positions, Obtain a weighted sum representing the contextual information of the location. The fully connected layer performs forward propagation on the context information to obtain the output of the layer. Specifically, the calculation formula of Multi-Head Attention is as follows:

Among them, d _k is the vector dimension, and Q, K, and V are all vectors. This calculation formula shows that for each query Q, Multi-Head Attention performs a weighted sum of all values V based on their similarity to all keys K.

The second multi-head attention mechanism network in the decoder is similar to the first multi-head attention mechanism network in the encoder, but when calculating the attention distribution, the second multi-head attention mechanism network in the decoder only considers the The position before the position, thus avoiding the problem of using future information in the decoder.

The training method of the load prediction model provided in the embodiment of the present application comprises an encoder and a decoder, and both the encoder and the decoder comprise a graph convolutional neural network, a feedforward neural network, and a multi-head attention mechanism network. Each graph structure in each training sample sequentially passes through the first graph convolutional neural network, the first feedforward neural network, the first multi-head attention mechanism network, and the second feedforward neural network in the encoder, and finally outputs the first road network spatiotemporal features after convolution calculation of each training sample, and inputs each graph structure in each true value sample sequentially into the second graph convolutional neural network and the third feedforward neural network to obtain the second road network spatial features after convolution calculation, and each training sample The third road network spatiotemporal characteristics of the corresponding multiple future moments and the second road network spatiotemporal characteristics of each true value sample are input into the fourth feedforward neural network to obtain the third road network spatiotemporal characteristics after convolution calculation and the second road network spatiotemporal characteristics after convolution calculation. According to the third road network spatiotemporal characteristics after convolution calculation and the second road network spatiotemporal characteristics after convolution calculation, the current loss value is calculated. When the current loss value does not meet the convergence condition, the model parameters of the load prediction model are adjusted, and the load prediction model after parameter adjustment is continued to be trained until the current loss value meets the convergence condition, and the final load prediction model is obtained, so as to perform load prediction based on the final load prediction model. It can be seen from this that the embodiment of the present application can analyze the load information of each server in the cloud computing platform from the spatial and temporal levels, and can capture long-term dependencies, so that the trained load prediction model can more accurately predict the load information of the server.

Based on the above method embodiment, another embodiment of the present application provides a server load prediction method, as shown in Figure 3. The method includes:

S310: Obtain the target data information of each server in the cloud computing platform at multiple recent historical moments.

The target data information at least includes load information, and may also include traffic and/or network performance. Traffic includes incoming traffic, outgoing traffic, and total traffic. Load information can include CPU usage, memory usage, etc. Network performance includes latency, bandwidth utilization, etc.

When you need to predict the load information of each server at multiple times in the future, you can first obtain the target data information at multiple recent historical times, such as the target data information at multiple times included in the last 10 minutes. Among them, the size of a moment can be determined according to actual needs. For example, a moment can refer to 1 minute or 5 minutes.

In order to improve the stability, generalization performance and robustness of the load prediction model, after obtaining the original data information of each server at multiple recent historical moments, you can first perform outlier detection on the original data information of each server. When outliers are detected in the original data information, the outliers are repaired. Finally, the original data information after the outlier repair can be normalized or standardized to obtain the target data information.

Among them, the method of detecting outliers on the original data information of each server includes: performing dimensionality reduction processing on the original data information based on the PCA algorithm, and using LOF to detect outliers on the reduced original data information, that is, using LOF calculation and identify outlier factor scores of original data information after dimensionality reduction, and determine outliers based on the scores. By combining the two methods, PCA and LOF, outliers can be detected more accurately and the robustness can be improved.

Methods for repairing outliers include: using a generative adversarial network to repair the outliers to obtain the repaired first data, and using the support vector regression algorithm to repair the outliers to obtain the repaired second data; The first data and the second data are weighted to obtain the final result after repairing the outliers.

Among them, a GAN model can be trained first, taking the original data information as input, and the goal is to generate synthetic data similar to the original data information. After training is completed, use the generator part to generate the repaired outlier data; at the same time, use support vector regression to generate the repaired outlier data. Finally, the repaired data obtained by these two methods are weighted and summed, and the final result after repair is generated based on the weighted results.

S320: For each historical moment, generate a graph structure for each historical moment based on the target data information of each server in the cloud computing platform.

After obtaining the target data information of each server in the cloud computing platform, a graph structure of each historical moment can be generated according to the topological structure of each server, so that each node in the graph structure represents a server, and the information carried by each node includes the target data information of the server at that historical moment.

S330: Inputting the graph structure of the most recent multiple historical moments into the load prediction model, and predicting the load information of each server at multiple future moments.

The load prediction model is trained according to the training method of the load prediction model provided in any of the above embodiments. By inputting the graph structure of the recent multiple historical moments into the pre-trained load prediction model, after being processed by the encoder and decoder in the load prediction model, the load information of each server at multiple future moments can be finally predicted.

The server load prediction method provided by the embodiment of the present application can first obtain the target data information of each server in the cloud computing platform at the most recent multiple historical moments, and then generate the graph structure of each historical moment based on the target data information of each server in the cloud computing platform for each historical moment, and finally input the graph structure of the most recent multiple historical moments into the load prediction model to predict the load information of each server at multiple moments in the future. The load prediction model includes an encoder and a decoder, and both the encoder and the decoder include a graph convolutional neural network, a feedforward neural network, and a multi-head attention mechanism network. The embodiment of the present application first converts the target data information into a graph structure including the server topology structure, and then extracts the road network spatial features from the spatial dimension based on the graph convolutional neural network, and then analyzes the dependency of each road network spatial feature from the time dimension based on the multi-head attention mechanism network, extracts the road network spatiotemporal features, and after each feature extraction, refines the extracted features through the feedforward neural network, which can accelerate the convergence of the load prediction model and improve the quality of the load prediction model. Therefore, the load prediction model trained by the embodiment of the present application can analyze the load information of each server in the cloud computing platform from the spatial and temporal levels, and can capture long-term dependencies, so that the trained load prediction model can more accurately predict the load information of the server.

Based on the above method embodiment, another embodiment of the present application provides a training device for a load prediction model. The load prediction model includes an encoder and a decoder. The encoder includes a first graph convolutional neural network, a third graph convolutional neural network, and a decoder. A feedforward neural network, a first multi-head attention mechanism network, and a second feedforward neural network. The decoder includes a second graph convolutional neural network, a third feedforward neural network, and a second multi-head attention mechanism network. , the fourth feedforward neural network, as shown in Figure 4, the device includes:

The acquisition unit 410 is used to acquire a training set, wherein the training set includes multiple training samples and a true value sample corresponding to each of the training samples, each of the training samples includes a graph structure of multiple consecutive historical moments, the true value sample corresponding to each of the training samples includes a graph structure of multiple future moments adjacent to the multiple consecutive historical moments, and the graph structure at each moment includes target data information of each of the multiple servers at that moment, and the target data information at least includes load information;

A first feature extraction unit 420, configured to input each of the graph structures in each of the training samples into the first graph convolutional neural network, respectively, and extract a first road network spatial feature of each of the graph structures in each of the training samples;

A second feature extraction unit 430, configured to input each of the graph structures in each of the true value samples into the second graph convolutional neural network, respectively, and extract a second road network spatial feature of each of the graph structures in each of the true value samples;

The first convolution calculation unit 440 is used to perform convolution calculation on each of the first road network spatial features of the first feedforward neural network to obtain the first road network spatial features after convolution calculation;

The second convolution calculation unit 450 is used to perform the convolution calculation of each of the second road network spatial features through the third feedforward neural network to obtain the second road network spatial features after convolution calculation;

The adding unit 460 is used to add corresponding time information to the first road network spatial feature after the convolution calculation at each moment;

The first attention processing unit 470 is used to input the first network spatial feature of each training sample after adding time information into the first multi-head attention mechanism network, and obtain the first network spatial feature of each training sample. The spatiotemporal characteristics of the first road network;

The third convolution calculation unit 480 is used to perform the convolution calculation of the first road network spatiotemporal feature of each training sample through the second feedforward neural network to obtain the first convolution calculation of each training sample. Road network spatiotemporal characteristics;

The second attention processing unit 490 is used to input the spatio-temporal features of the first road network calculated by convolution and the plurality of second road network spatial features in the corresponding true value samples after adding time information into the second plurality of In the head attention mechanism network, obtain the spatio-temporal characteristics of the third network at multiple times in the future corresponding to each training sample, and the spatio-temporal characteristics of the second network of each true value sample;

The fourth convolution calculation unit 4100 is used to input the spatio-temporal features of the third road network at multiple times in the future corresponding to each training sample, and the spatio-temporal features of the second road network for each of the true value samples into the third Four feedforward neural networks are used to obtain the spatio-temporal characteristics of the third network after convolution calculation and the spatio-temporal characteristics of the second network after convolution calculation;

The adjustment training unit 4110 is used to calculate the current loss value based on the spatio-temporal characteristics of the third road network after convolution calculation and the spatio-temporal characteristics of the second road network after convolution calculation. When the current loss value does not meet the convergence condition, the The model parameters of the load prediction model are adjusted, and the load prediction model after parameter adjustment is continued to be trained until the current loss value meets the convergence condition, and the final load prediction model is obtained.

In a possible implementation, the acquiring unit 410 includes:

An outlier detection module, used for performing outlier detection on the original data information of each server;

A repair module, used for repairing the abnormal value when an abnormal value is detected in the original data information;

The processing module is used to perform normalization or standardization operations on the original data information after the outliers are repaired to obtain the target data information.

In a possible implementation, the outlier detection module is used to perform dimensionality reduction processing on the original data information based on the principal component analysis PCA algorithm, and use the local outlier factor LOF to perform dimensionality reduction on the original data information after dimensionality reduction. Perform outlier detection.

In a possible implementation, the repair module is configured to use a generative adversarial network to repair the outliers and obtain the repaired first data, and to use a support vector regression algorithm to repair the outliers and obtain the repaired data. perform a weighted calculation on the first data and the second data to obtain the final result after repairing the outliers.

In a possible implementation, the target data information also includes: traffic and/or network performance.

The training device of the load prediction model provided by the embodiment of the present application. The load prediction model includes an encoder and a decoder, and both the encoder and the decoder include a graph convolutional neural network, a feedforward neural network, and a multi-head attention mechanism network. Each graph structure in each training sample sequentially passes through the first graph convolutional neural network, the first feedforward neural network, the first multi-head attention mechanism network and the second feedforward neural network in the encoder, and finally outputs each The spatio-temporal characteristics of the first network after the convolution calculation of the training sample are input into the second graph convolutional neural network and the third feedforward neural network in sequence for each graph structure in each true value sample to obtain the second network after the convolution calculation. For the spatial characteristics of the road network, input the spatio-temporal features of the third road network at multiple times in the future corresponding to each training sample, and the spatio-temporal features of the second road network for each true value sample into the fourth feedforward neural network to obtain the convolution calculation The current loss value is calculated based on the spatio-temporal characteristics of the third road network after convolution calculation and the spatio-temporal characteristics of the second road network after convolution calculation. , when the current loss value does not meet the convergence condition, adjust the model parameters of the load prediction model, and continue to train the load prediction model after parameter adjustment, until the current loss value meets the convergence condition, and obtain the final load prediction model. In order to perform load prediction based on the final load prediction model. It can be seen from this that the embodiments of the present application can analyze the load information of each server in the cloud computing platform from the spatial and temporal levels, and can capture long-term dependencies, so that the trained load prediction model can more accurately predict the load information of the server. .

Based on the above method embodiment, another embodiment of the present application provides a server load prediction device, as shown in Figure 5, the device includes:

The acquisition unit 510 is used to acquire the target data information of each server in the cloud computing platform at multiple recent historical moments, where the target data information at least includes load information;

A generating unit 520, configured to generate, for each historical moment, a graph structure of each historical moment based on the target data information of each server in the cloud computing platform;

The prediction unit 530 is configured to input the graph structure of multiple recent historical moments into a load prediction model, and predict the load information of each server at multiple moments in the future, wherein the load prediction model is provided according to any of the above embodiments. It is trained using the load prediction model training method.

The server load prediction device provided by the embodiment of the present application can first obtain the target data information of each server in the cloud computing platform at multiple recent historical moments, and then for each historical moment, based on the target data information of each server in the cloud computing platform Generate the graph structure of each historical moment, and finally input the graph structure of multiple recent historical moments into the load prediction model to predict the load information of each server at multiple times in the future. The load prediction model includes an encoder and a decoder, and both the encoder and the decoder include a graph convolutional neural network, a feedforward neural network, and a multi-head attention mechanism network. In this embodiment, the target data information is first converted into a server-based Topological graph structure, then extract the spatial features of the road network from the spatial dimension based on the graph convolutional neural network, and then analyze the dependence of the spatial features of each road network from the time dimension based on the multi-head attention mechanism network, and extract the spatiotemporal features of the road network, and After each feature is extracted, the extracted features are refined through a feedforward neural network, which can speed up the convergence of the load prediction model and improve the quality of the load prediction model. Therefore, the load prediction model trained in the embodiments of this application can analyze the load information of each server in the cloud computing platform from the spatial and temporal levels, and can capture long-term dependencies, so that the trained load prediction model can predict the server more accurately. load information.

Based on the above method embodiments, another embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method described in any of the above embodiments is implemented.

Based on the above method embodiment, another embodiment of the present application provides an electronic device or computer device, including:

one or more processors;

The processor is coupled to a storage device for storing one or more programs;

Wherein, when the one or more programs are executed by the one or more processors, the electronic device or computer device is caused to implement the method described in any of the above embodiments.

Based on the above embodiments, another embodiment of the present application provides a computer program product. The computer program product contains instructions. When the instructions are run on a computer or processor, the computer or processor causes the computer or processor to perform any of the above. The method described in the embodiment.

The above device embodiments correspond to the method embodiments and have the same technical effects as the method embodiments. For detailed description, please refer to the method embodiments. The device embodiment is obtained based on the method embodiment. For specific description, please refer to the method embodiment section and will not be described again here. Those of ordinary skill in the art can understand that the accompanying drawing is only a schematic diagram of an embodiment, and the modules or processes in the accompanying drawing are not necessarily necessary for implementing the present application.

Those of ordinary skill in the art can understand that the modules in the device in the embodiment may be distributed in the device in the embodiment according to the description of the embodiment, or may be correspondingly changed and located in one or more devices different from this embodiment. The modules of the above embodiments can be combined into one module, or further divided into multiple sub-modules.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims (10)

1. A method of training a load prediction model, the load prediction model comprising an encoder and a decoder, the encoder comprising a first graph roll-up neural network, a first feed-forward neural network, a first multi-head attention mechanism network, a second feed-forward neural network, the decoder comprising a second graph roll-up neural network, a third feed-forward neural network, a second multi-head attention mechanism network, a fourth feed-forward neural network, the method comprising:

Acquiring a training set, wherein the training set comprises a plurality of training samples and true value samples corresponding to each training sample, each training sample comprises a graph structure of a plurality of continuous historical moments, each true value sample corresponding to each training sample comprises a graph structure of a plurality of future moments adjacent to the plurality of continuous historical moments, each graph structure of each moment comprises target data information of each server in a plurality of servers at the moment, and the target data information at least comprises load information;

respectively inputting each graph structure in each training sample into the first graph convolution neural network, extracting a first road network space characteristic of each graph structure in each training sample, respectively inputting each graph structure in each truth sample into the second graph convolution neural network, and extracting a second road network space characteristic of each graph structure in each truth sample;

the convolution calculation of each first path network space feature through the first feedforward neural network is carried out to obtain a first path network space feature after the convolution calculation, and the convolution calculation of each second path network space feature through the third feedforward neural network is carried out to obtain a second path network space feature after the convolution calculation;

Adding corresponding time information to the first road network space characteristics calculated by convolution at each moment, and inputting the first road network space characteristics of each training sample after adding the time information into the first multi-head attention mechanism network to obtain first road network space-time characteristics of each training sample;

the first path network space-time characteristics of each training sample are subjected to convolution calculation of the second feedforward neural network, so that the first path network space-time characteristics of each training sample after convolution calculation are obtained;

inputting the first road network space-time characteristics after convolution calculation and a plurality of second road network space-time characteristics in corresponding truth value samples after time information addition into the second multi-head attention mechanism network to obtain third road network space-time characteristics of a plurality of future moments corresponding to each training sample and second road network space-time characteristics of each truth value sample;

inputting the third path network space-time characteristics of a plurality of future moments corresponding to each training sample and the second path network space-time characteristics of each truth sample into the fourth feedforward neural network to obtain the third path network space-time characteristics after convolution calculation and the second path network space-time characteristics after convolution calculation;

According to the third path network space-time characteristic after convolution calculation and the second path network space-time characteristic after convolution calculation, calculating a current loss value, adjusting model parameters of the load prediction model when the current loss value does not meet a convergence condition, and continuously training the load prediction model after parameter adjustment until the current loss value meets the convergence condition, and obtaining a final load prediction model.

2. The method according to claim 1, wherein the method for acquiring the target data information of each of the servers comprises:

detecting abnormal values of the original data information of each server;

when detecting that an abnormal value exists in the original data information, repairing the abnormal value;

and carrying out normalization or standardization operation on the original data information after the abnormal value is repaired to obtain the target data information.

3. The method of claim 2, wherein said anomaly value detection of raw data information for each of said servers comprises:

and performing dimension reduction processing on the original data information based on a Principal Component Analysis (PCA) algorithm, and detecting an outlier of the dimension reduced original data information by utilizing a Local Outlier Factor (LOF).

4. The method of claim 2, wherein the repairing the outlier comprises:

repairing the abnormal value by using a generated countermeasure network to obtain repaired first data, and repairing the abnormal value by using a support vector regression algorithm to obtain repaired second data;

and carrying out weighted calculation on the first data and the second data to obtain a final result after repairing the abnormal value.

5. The method of any of claims 1-4, wherein the target data information further comprises: traffic and/or network performance.

6. A method for predicting load of a server, the method comprising:

acquiring target data information of each server in the cloud computing platform at a plurality of latest historical moments, wherein the target data information at least comprises load information;

generating, for each historical moment, a graph structure of the each historical moment based on the target data information of the respective server in the cloud computing platform;

the graph structure of the latest historical moments is input into a load prediction model, and the load information of each server of the future moments is predicted, wherein the load prediction model is trained according to the method of any one of claims 1-5.

7. A training apparatus for a load prediction model, the load prediction model comprising an encoder and a decoder, the encoder comprising a first graph roll-up neural network, a first feed-forward neural network, a first multi-head attention mechanism network, a second feed-forward neural network, the decoder comprising a second graph roll-up neural network, a third feed-forward neural network, a second multi-head attention mechanism network, a fourth feed-forward neural network, the apparatus comprising:

the training set comprises a plurality of training samples and true value samples corresponding to each training sample, each training sample comprises a graph structure of a plurality of continuous historical moments, each true value sample corresponding to each training sample comprises a graph structure of a plurality of future moments adjacent to the plurality of continuous historical moments, each graph structure of each moment comprises target data information of each server in a plurality of servers at the moment, and the target data information at least comprises load information;

the first feature extraction unit is used for respectively inputting each graph structure in each training sample into the first graph convolution neural network and extracting the first road network space feature of each graph structure in each training sample;

The second feature extraction unit is used for respectively inputting each graph structure in each truth value sample into the second graph convolution neural network and extracting a second road network space feature of each graph structure in each truth value sample;

the first convolution calculation unit is used for carrying out convolution calculation on each first path network space feature through the first feedforward neural network to obtain a first path network space feature after convolution calculation;

the second convolution calculation unit is used for carrying out convolution calculation on each second road network spatial feature through the third feedforward neural network to obtain a second road network spatial feature after convolution calculation;

the adding unit is used for adding corresponding time information to the first road network space characteristics after convolution calculation at each moment;

the first attention processing unit is used for inputting the first road network space characteristics of each training sample after time information is added into the first multi-head attention mechanism network to obtain the first road network space-time characteristics of each training sample;

the third convolution calculation unit is used for carrying out convolution calculation on the first path network space-time characteristics of each training sample through the second feedforward neural network to obtain first path network space-time characteristics of each training sample after convolution calculation;

The second attention processing unit is used for inputting the first path network space-time characteristics after convolution calculation and a plurality of second path network space-time characteristics in the corresponding truth value samples after time information addition into the second multi-head attention mechanism network to obtain third path network space-time characteristics of a plurality of future moments corresponding to each training sample and second path network space-time characteristics of each truth value sample;

a fourth convolution calculation unit, configured to input a third path network space-time feature of a future multiple moments corresponding to each training sample and a second path network space-time feature of each truth sample into the fourth feedforward neural network, to obtain a third path network space-time feature after convolution calculation and a second path network space-time feature after convolution calculation;

the adjustment training unit is used for calculating a current loss value according to the third path network space-time characteristic after convolution calculation and the second path network space-time characteristic after convolution calculation, adjusting the model parameters of the load prediction model when the current loss value does not meet the convergence condition, and continuing to train the load prediction model after parameter adjustment until the current loss value meets the convergence condition, so as to obtain a final load prediction model.

8. A load predicting apparatus of a server, the apparatus comprising:

the cloud computing platform comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring target data information of each server in the cloud computing platform at a plurality of latest historical moments, and the target data information at least comprises load information;

a generation unit configured to generate, for each history time, a graph structure of each history time based on the target data information of each server in the cloud computing platform;

the prediction unit is configured to input the graph structures of the most recent historical moments into a load prediction model, and predict load information of each server of the plurality of future moments, where the load prediction model is trained according to the method of any one of claims 1-5.

9. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any one of claims 1-5 or the method according to claim 6.

10. An electronic device, the electronic device comprising:

one or more processors;

the processor is coupled with a storage device for storing one or more programs;

The one or more programs, when executed by the one or more processors, cause the electronic device to perform the method of any of claims 1-5 or the method of claim 6.