CN113220450B - Load prediction method, resource scheduling method and device for cloud-side multi-data center - Google Patents

Abstract

The invention discloses a load prediction method oriented to multiple data centers in the cloud, including the following processes: acquiring a log record file that records the resource usage of a virtual machine at each time point, and extracting the required feature quantity data and historical load data therefrom; and Convert feature quantity data and historical load data into corresponding input feature sequences and historical load vectors; use pre-built neural network models to calculate nonlinear components of load prediction; use pre-built autoregressive models to calculate linear components of load prediction ; Integrate the nonlinear and linear components of load prediction to get the final load prediction result. The invention comprehensively considers the time-varying linear trend and nonlinear characteristics of the load sequence in the multi-data center environment, and combines the neural network model with the statistical learning method of the autoregressive model, which can effectively improve the prediction accuracy of future loads.

Description

Load prediction method, resource scheduling method and device for cloud multi-data center

technical field

The invention specifically relates to a load prediction method oriented to cloud multi-data centers, and also relates to a resource scheduling method oriented to cloud multi-data centers, belonging to the technical field of cloud computing and data mining.

Background technique

The growing computing demand of cloud computing technology has prompted the continuous expansion of the scale of cloud data centers. It is expected that by the end of 2022, the domestic data center business market will grow to 320 billion yuan, and its scale structure will also change from a single data center to multiple cloud data centers. In order to achieve green energy-saving development, the data center needs to have the decision-making ability to dynamically adjust its internal resource allocation, and integrate computing resources to achieve service flexibility. Effective load prediction is a prerequisite for realizing resource elastic allocation, which can provide decision-making reference for high-quality expansion schemes. Therefore, based on the current multi-data center environment in the cloud, it is of great significance to establish an appropriate load prediction model combined with load characteristics.

Workload is a time-series problem that is closely related to context, yet most studies of load forecasting are still stagnant at the scope of a single data center. Compared with a traditional single data center, multi-data center resource scheduling and expansion capabilities are stronger, business deployment is more flexible, and load changes are more prominently affected by user behavior. As a result, the computing resource requirements required by each of them are dynamically changed, and the load fluctuation trend of different data centers is also quite different. Therefore, the load forecasting for multiple data centers in the cloud needs to be modeled and analyzed in combination with the specific conditions of each data center. , can not rely solely on the original single data center-oriented algorithm.

In view of this, it is indeed necessary to propose a load prediction method and resource allocator system for cloud multi-data centers to solve the above problems.

SUMMARY OF THE INVENTION

The purpose of the present invention is to overcome the deficiencies in the prior art, and provide a load prediction method oriented to cloud multi-data centers, which combines a neural network model and an autoregressive model to predict the nonlinear component and linearity of future load changes on the server. components to improve the accuracy of load prediction results.

In order to solve the above technical problems, the technical solutions of the present invention are as follows.

In a first aspect, the present invention provides a load prediction method for multiple data centers in the cloud, including the following processes:

Obtain the log record file that records the resource usage of the virtual machine at each point in time, extract the required feature data and historical load data from it, and convert the feature data and historical load data into the corresponding input feature sequence and historical load vector;

Based on the obtained input feature sequence and historical load vector, the nonlinear component of load prediction is calculated by using a pre-built neural network model;

Based on the obtained historical load vector, the linear component of load prediction is calculated by using a pre-built autoregressive model;

The nonlinear and linear components of load prediction are integrated to obtain the final load prediction result.

Optionally, the feature quantities include: sampling and recording time, virtual machine core number configuration, CPU capacity, virtual machine memory configuration capacity, effective usage of virtual machine memory, disk read throughput, disk write throughput, network Receive throughput and network transmission throughput; the load refers to the effective CPU usage.

Optionally, converting the feature quantity data into a corresponding input feature sequence includes:

The feature quantity data is segmented by sliding window to form a time series with a fixed time step as the input feature sequence.

Optionally, the neural network model includes: an encoder, a decoder, and a multi-layer perceptron network; wherein, the input of the encoder is the collected input feature sequence, and the input of the decoder is the adaptive extraction of the output of the upper-layer encoder. The input feature sequence of the multi-layer perceptron network is the text vector output by the decoder.

Optionally, based on the obtained input feature sequence and historical load vector, use a pre-built neural network model to calculate and obtain the nonlinear component of load prediction, including:

和细胞状态s_t-1作为输入参数，使用：The encoder module includes an input attention layer, a softmax layer, and an LSTM neural network layer. The output of each layer is the input of the next layer; in the data loop update of the encoder module, the hidden layer state output by the previous LSTM unit needs to be updated.

and the cell state s _t-1 as input parameters, use:

h _t =f ₁ (h _t-1 ,X _t )

表示m维度的实数向量空间，X_t表示在t时刻的输入特征序列；Represents an update calculation process; where f ₁ represents the LSTM unit,

Represents a real vector space of m dimension, X _t represents the input feature sequence at time t;

In an LSTM unit, at each time step there is a computation:

分别是施加给r_t和h_t-1的权重矩阵，

表示4m×d_r维度的实数向量空间，

表示4m×m维度的实数向量空间，m是隐藏层维度，d_r是输入r_t的向量维度；r_t是t时刻的输入；h_t-1是t-1时刻输出的隐藏状态向量；

是当前时刻神经网络模型输出的候选单元状态向量；sigmoid和 tanh分别表示不同的激活函数；Among them, f _t , i _t , o _t represent forgetting gate, input gate and output gate respectively;

are the weight matrices applied to r _t and h _t-1 , respectively,

represents a real vector space of 4m × d _r dimensions,

Represents a real vector space of 4m×m dimensions, m is the hidden layer dimension, d _r is the vector dimension of the input r _t ; r _t is the input at time t; h _t-1 is the hidden state vector output at time t-1;

is the candidate unit state vector output by the neural network model at the current moment; sigmoid and tanh represent different activation functions respectively;

Then, the weight corresponding to each input feature sequence can be obtained through the input attention mechanism

是一个中间变量，无具体实际含义，

和

是注意力机制模型中需要学习的参数，

表示T维度的实数向量空间，

表示 T×2m维度的实数向量空间，

表示T×T维度的实数向量空间；tanh表示双曲正切激活函数，exp表示指数函数；

的计算是通过softmax层进行处理；in,

is an intermediate variable with no specific actual meaning,

and

is the parameter that needs to be learned in the attention mechanism model,

represents a real vector space of dimension T,

represents a real vector space of dimension T × 2m,

Represents the real vector space of T×T dimension; tanh represents the hyperbolic tangent activation function, and exp represents the exponential function;

The calculation is processed through the softmax layer;

Through the attention weight, the adaptively extracted input feature sequence can be obtained:

In turn, the hidden layer state of the LSTM unit can be updated as:

表示在t时刻的自适应输入特征序列，从而让编码器能够有选择性地专注于更加重要的相关输入特征，而非平等对待所有的输入特征序列，实现了发掘特征序列之间的相互依赖关系；use

Represents the adaptive input feature sequence at time t, so that the encoder can selectively focus on more important relevant input features, rather than treating all input feature sequences equally, and realizes the interdependence between feature sequences. ;

In the decoder module, because in the traditional encoder-decoder model, when the input sequence is too long, its representation ability decreases, and the model effect deteriorates rapidly. Therefore, the temporal attention mechanism is used in the decoding layer of the model to adaptively select relevant correlations. the state of the hidden layer.

和细胞状态

作为输入参数，其中z代表编码器中隐藏层的维度，

表示z维度的实数向量空间，其重要性权重

的公式推导过程与输入的特征量序列关注度

的计算过程相同，

代表第k个编码器隐藏状态h_k对最终预测的重要性大小。接着解码器将所有的编码器隐藏层状态按照权重求和得到文本向量：Similar to the method in the encoder module, the attention mechanism in the decoder module also requires the hidden layer state output from the previous LSTM unit

and cell state

as an input parameter, where z represents the dimension of the hidden layer in the encoder,

Represents a real vector space in z dimension, its importance weights

The formula derivation process of and the attention of the input feature sequence

The calculation process is the same,

Represents the importance of the kth encoder hidden state h _k to the final prediction. Then the decoder sums all the encoder hidden layer states according to the weights to obtain the text vector:

Combining the historical load data {L _TP ,L _T-P+1 ,...,L _T-1 } and the obtained text vector, after vector splicing and linear change, the adaptively extracted decoding layer input can be obtained:

为t-1时刻的负载与计算得到的文本向量的拼接，m是前面所述的解码器中的隐藏层维度，

表示m+1维度的实数向量空间。

和

是线性变换过程中待学习的参数。接着利用计算得到的

更新解码器隐藏层状态：in,

is the splicing of the load at time t-1 and the calculated text vector, m is the hidden layer dimension in the decoder described above,

Represents a real vector space of dimension m+1.

and

is the parameter to be learned in the linear transformation process. Then use the calculated

Update the decoder hidden layer state:

Among them, f ₂ is a nonlinear activation function of an LSTM unit, and its specific update calculation method is consistent with f ₁ .

The output layer is composed of a multi-layer perceptron. The final hidden layer state output by the encoding and decoding model, namely {d _TP , d _T-P+1 ,..., d _T-1 }, is used as input, and the three-layer perceptron network is used as input. The output is the final model prediction result, where PReLU is used as the activation function in the first two layers of the multilayer perceptron:

f ₃ =max(μd _t ,d _t )

where _f3 represents the PReLU activation function and μ is a parameter that is only updated during training. The PReLU activation function avoids the problem that some parameters cannot be updated. The activation function of the last layer of perceptron is a sigmoid function to ensure that the prediction results can be limited within a reasonable range.

Finally, after T time steps, the nonlinear part of the final load prediction result is obtained

代表在经历了T个时间步后对于负载预测的非线性部分，我们用T表示时间步大小，non表示上述非线性的标识，F_non用于表示上述整个负载预测的非线性计算过程，{L_T-P,...,L_T-1}表示T时间步之前P-1个时间步的负载大小， {X_T-P,...,X_T}表示T时间步前P个时间步的输入特征序列。

是隐藏层状态和文本向量的拼接向量，

表示z+m维度的实数向量空间，参数W_y和b_w实现了将拼接向量映射为解码层隐藏层状态的尺寸，其中W_y表示拼接向量在解码器计算过程中对应的非线性映射权重，b_w表示偏置值，其计算实现过程由计算机程序实现，同理

和u_w分别代表输出层中的非线性映射权重和偏置值，使用F_non代表上述过程的神经网络预测函数，

表示计算结果即最终预测结果的非线性组成部分。in,

Represents the nonlinear part of load prediction after T time steps, we use T to represent the time step size, non to represent the above nonlinear identifier, F _non to represent the nonlinear calculation process of the entire load prediction above, {L _TP ,...,L _T-1 } represents the load size of P-1 time steps before T time step, {X _TP ,...,X _T } represents the input feature sequence of P time steps before T time step .

is the concatenation vector of the hidden layer state and the text vector,

Represents the real vector space of z+m dimension, and the parameters W _y and b _w realize the size of mapping the splicing vector to the hidden layer state of the decoding layer, where W _y represents the corresponding nonlinear mapping weight of the splicing vector in the decoder calculation process, b _w represents the bias value, and its calculation and realization process is realized by a computer program.

and u _w represent the nonlinear mapping weight and bias value in the output layer, respectively, and use F _non to represent the neural network prediction function of the above process,

Represents the nonlinear component of the calculation result, which is the final prediction result.

Optionally, the specific calculation formula of the autoregressive model is:

表示计算结果即最终预测结果的线性组成部分。Among them, {L _TP ,L _T-P+1 ,...,L _T-1 } is the historical data, ε _T is the random disturbance variable, and λ _t is the weight corresponding to each moment. In the design of the autoregressive model, initialization and automatic numerical update are realized, and F _linear is used to represent the autoregressive prediction function of the above process.

Represents the linear component of the calculation result, which is the final prediction result.

In a second aspect, the present invention also provides a load prediction device oriented to multiple data centers in the cloud, including:

The data processing module is used to obtain the log record file that records the resource usage of the virtual machine at each time point, extract the required feature data and historical load data from it, and convert the feature data and historical load data into corresponding input features sequence and historical load vectors;

The nonlinear component prediction module is used to calculate the nonlinear component of load prediction by using the pre-built neural network model based on the obtained input feature sequence and historical load vector;

The linear component prediction module is used to calculate the linear component of the load prediction by using the pre-built autoregressive model based on the obtained historical load vector;

The forecasting result calculation module integrates the nonlinear and linear components of load forecasting to obtain the final load forecasting result.

In a third aspect, the present invention also provides a resource scheduling method for multiple data centers in the cloud, including the following processes:

Calculate the load prediction result of the virtual machines on each server in the cluster in the cloud multi-data center environment based on the above method;

A corresponding resource scheduling policy is generated based on the load prediction results of the virtual machines on each server.

In a fourth aspect, the present invention also provides a resource allocator, comprising:

A load prediction module, configured to calculate and obtain the load prediction result of the virtual machines on each server in the cluster in the cloud multi-data center environment based on the above method;

The resource scheduling module is used for generating corresponding resource scheduling policies based on the load prediction results of the virtual machines on each server.

Compared with the prior art, the beneficial effects achieved by the present invention are: the present invention comprehensively considers the time-varying linear trend and nonlinear characteristics of the load sequence under the multi-data center environment, and combines the LSTM-based neural network method with the autoregressive method. The combination of statistical learning methods of the model can effectively improve the prediction accuracy of future load testing.

Description of drawings

Fig. 1 is the structural representation of the system of the present invention;

Fig. 2 is the flow chart of the inventive method;

Fig. 3 is the principle schematic diagram of the encoder-decoder module in the neural network model;

Fig. 4 is a schematic diagram of a multilayer perceptron output network in a neural network model;

Fig. 5 is the concrete principle structure block diagram of the system of the present invention;

Figure 6 shows the actual load variation trend of a server in the data center over a period of time.

Detailed ways

The present invention will be further described below in conjunction with the accompanying drawings. The following examples are only used to illustrate the technical solutions of the present invention more clearly, and cannot be used to limit the protection scope of the present invention.

Example 1

The present invention designs a load prediction method and resource scheduling method oriented to multiple data centers in the cloud. The system takes the log record files of each virtual machine obtained by the data center in the cloud as input, and preprocesses the data to predict the linearity and non-linearity of the load. The linear change finally enables the system to generate the corresponding resource allocation scheduling strategy according to the load prediction result.

A load prediction method for cloud multi-data centers of the present invention, as shown in FIG. 2 , includes the following steps:

Step 1, first obtain the log record file of the virtual machine on the server in the cluster from the cloud data center. The log file of the virtual machine contains the usage of various resources occupied by the virtual machine at each time point. Extract the required feature quantity from the log record file and convert it into a time series data format that the system can recognize and process;

Data collection and processing are for all virtual machines. During this process, a large number of virtual machine log records will be collected. The processing and analysis methods are the same. Therefore, in this embodiment, a virtual machine is used as an example to process log records. file and perform load forecasting.

In order to simplify the complex information in the massive log record files, the non-time-series information that has no effect on load forecasting is eliminated. The invention firstly specifies the types of feature quantities required by the model in the preprocessing stage, and standardizes the time series data format. Therefore, it is convenient to learn richer structural information and contextual connections in the various data that affect the load.

The specific process of preprocessing the log record file is as follows:

1) First, clean the collected log record files, extract the required feature data, discard the rest of the record data irrelevant to load prediction, and only retain the virtual machine resource usage at each point in time. If there is a local data anomaly, choose the average value of the adjacent data before and after the field to replace the abnormal data. When the sequence of a certain segment is missing two or more, considering the complex nonlinear relationship that may exist between the features of the data center, In order to avoid the impact of simple data processing on the training accuracy, this abnormal data with missing data is discarded.

2) The final collected data should include but not limited to the following features: sampling and recording time, configuration of virtual machine cores, CPU capacity, CPU effective usage, virtual machine memory configuration capacity, virtual machine memory effective usage, disk Read throughput, disk write throughput, network receive throughput, network transmit throughput. The effective CPU usage is used as the target of load prediction, and the rest of the features are used as the input data of the model.

The recording time 1970.01.01 of the feature quantity is the default start time of the computer, the unit MHZ stands for megahertz, which is one of the fluctuation frequency units, and the unit KB/S stands for the number of kilobytes that can be processed per second.

3) Then use the sliding window to segment the collected feature quantity data to form a time series curve with a fixed time step, which is then used as the input feature sequence of the model.

来代表某一台虚拟机日志文件整理完后得到的输入特征序列，其中，n代表负载预测输入的特征维度， P代表输入特征序列的时间步长，T代表预测的目标时刻，

表示一个n×P维度的实数空间。设置

每一个X^k均代表作为输入特征之一的时间步长为P的特征序列，另外，引入

代表在某个时刻t的含有n个输入特征的负载预测的输入特征序列。use

to represent the input feature sequence obtained after finishing the log file of a virtual machine, where n represents the feature dimension of the load prediction input, P represents the time step of the input feature sequence, T represents the predicted target time,

Represents a real space of n×P dimensions. set up

Each X ^k represents a feature sequence with time step P as one of the input features. In addition, we introduce

represents a sequence of input features for load prediction with n input features at some time t.

表示在T时刻的云数据中心的CPU的有效使用量即当前的瞬时，也是我们的预测目标。In addition, using L={L _TP ,L _T-P+1 ,...,L _T-1 } to represent the historical load data vector, use

It represents the effective usage of the CPU of the cloud data center at time T, that is, the current instant, which is also our prediction target.

To sum up, after preprocessing the log record file, the required input feature sequence X={X ¹ ,X ² ,...,X ⁿ } ^P and the historical load data vector L={L _TP ,L are obtained _T-P+1 ,...,L _T-1 }.

In step 2, the input feature sequence and historical load data vector obtained after the processing in step 1 are respectively introduced into the pre-designed neural network model and the autoregressive model, and the load prediction result at the next moment is output.

The composition of the neural network model includes encoder, decoder and multilayer perceptron network. Among them, the input of the encoder is the collected input feature sequence, the input of the decoder is the adaptively extracted input feature sequence output by the upper encoder, and the input of the multilayer perceptron network is the text vector output by the decoder. The neural network module is used to extract the interdependence between the feature sequences, analyze the nonlinear change trend between the feature quantities, and finally output the nonlinear component of the load prediction result. The attention mechanism is embedded in the encoder and decoder to explore the weight of the feature data on the load, and to analyze the previous load sequence and the impact of each feature on the load.

The structure and specific processing process of the neural network are shown in Figure 3 and Figure 4. First, all variables and symbols appearing in Figure 3 and Figure 4 are explained: {X ¹ , X ² ,...,X ⁿ } ^P represents The collected input feature sequence, because it is the data collected at each time node, can also be called a time series. The time series can be split so that {x ¹ _TP ,x ¹ _T-P+1 ,...,x ¹ _T } to {x ⁿ _TP ,x ⁿ _T-P+1 ,...,x The n feature vectors represented by ⁿ _T } are composed of a single feature quantity, where n represents the feature dimension of the load prediction input, P represents the time step of the input feature sequence, and T represents the predicted target time.

代表第k条输入特征序列在t时刻的数值，

代表在编码器中得到的第k条输入特征序列在t时刻的数值对应的权重大小。

代表在t时刻第k个自适应提取得到输入向量。LSTM代表长短期记忆神经网络，它是编码器解码器模型中进行模型参数更新的重要组件。d_t代表解码器中产生的t时刻对应的隐藏层张量数据。

代表在解码器中得到的第i个编码器的隐藏层状态在t时刻的数值对应的权重。

是解码器将所有的编码器隐藏状态按照权重求和得到的文本向量，

是一个求和符号代表将

到

的所有值进行相加。L_t代表t时刻对应的负载即CPU的有效使用，

表示在T时刻神经网络模型的得到了负载预测结果的非线性组成部分。h _t represents the hidden layer tensor data corresponding to a certain time t generated in the encoder of the neural network model. The softmax function can also be called a normalized exponential function, and its purpose is to make the sum of all hidden layer weights generated by the encoder equal to 1.

represents the value of the kth input feature sequence at time t,

Represents the weight corresponding to the value of the kth input feature sequence obtained in the encoder at time t.

Represents the input vector obtained by the k-th adaptive extraction at time t. LSTM stands for Long Short-Term Memory Neural Network, and it is an important component in the encoder-decoder model for model parameter updating. d _t represents the hidden layer tensor data corresponding to time t generated in the decoder.

Represents the weight corresponding to the value of the hidden layer state of the ith encoder obtained in the decoder at time t.

is the text vector obtained by the decoder summing all the hidden states of the encoder according to the weights,

is a summation symbol representing the

arrive

All values of . L _t represents the load corresponding to the time t, that is, the effective use of the CPU,

Represents the nonlinear component of the neural network model that obtains the load prediction result at time T.

和细胞状态s_t-1作为输入参数，使用：In the encoder module, an input attention mechanism is introduced to achieve adaptive weighting to the input feature sequence. The encoder module includes a total of input attention layer, softmax layer, and LSTM neural network layer as shown in Figure 3. The output of each layer is the input of the next layer. The hidden layer state output from the previous LSTM unit needs to be updated in the encoder module data loop update

and the cell state s _t-1 as input parameters, use:

h _t =f ₁ (h _t-1 ,X _t )

表示m维度的实数向量空间，X_t表示在t时刻的输入特征序列。Represents a calculation process for this update. where f ₁ represents the LSTM unit described,

represents a real vector space of dimension m, and X _t represents the input feature sequence at time t.

In an LSTM unit, at each time step there is a computation:

分别是施加给r_t和h_t-1的权重矩阵，

表示4m×d_r维度的实数向量空间，

表示4m×m维度的实数向量空间，m是隐藏层维度，d_r是输入r_t的向量维度；r_t是t时刻的输入；h_t-1是t-1时刻输出的隐藏状态向量；

是当前时刻神经网络模型输出的候选单元状态向量；sigmoid和tanh 分别表示不同的激活函数。Among them, f _t , i _t , o _t represent forgetting gate, input gate and output gate respectively;

are the weight matrices applied to r _t and h _t-1 , respectively,

represents a real vector space of 4m × d _r dimensions,

Represents a real vector space of 4m×m dimensions, m is the hidden layer dimension, d _r is the vector dimension of the input r _t ; r _t is the input at time t; h _t-1 is the hidden state vector output at time t-1;

is the candidate unit state vector output by the neural network model at the current moment; sigmoid and tanh represent different activation functions respectively.

Then, the weight corresponding to each input feature sequence can be obtained through the input attention mechanism

是一个中间变量，无具体实际含义，

和

是注意力机制模型中需要学习的参数，

表示T维度的实数向量空间，

表示T×2m维度的实数向量空间，

表示T×T维度的实数向量空间。tanh表示双曲正切激活函数，exp表示指数函数。

的计算是通过softmax层进行处理。通过关注度权重大小，可以得到自适应提取的输入特征序列：in,

is an intermediate variable with no specific actual meaning,

and

is the parameter that needs to be learned in the attention mechanism model,

represents a real vector space of dimension T,

represents a real vector space of dimension T × 2m,

Represents a real vector space of T×T dimension. tanh represents the hyperbolic tangent activation function, and exp represents the exponential function.

The computation is processed through a softmax layer. Through the attention weight, the adaptively extracted input feature sequence can be obtained:

In turn, the hidden layer state of the LSTM unit can be updated as:

表示在t时刻的自适应输入特征序列，从而让编码器能够有选择性地专注于更加重要的相关输入特征，而非平等对待所有的输入特征序列，实现了发掘特征序列之间的相互依赖关系。use

Represents the adaptive input feature sequence at time t, so that the encoder can selectively focus on more important relevant input features, rather than treating all input feature sequences equally, and realizes the interdependence between feature sequences. .

In the decoder module, because in the traditional encoder-decoder model, when the input sequence is too long, its representation ability decreases, and the model effect deteriorates rapidly. Therefore, the temporal attention mechanism is used in the decoding layer of the model to adaptively select relevant correlations. the state of the hidden layer.

和细胞状态

作为输入参数，其中z代表编码器中隐藏层的维度，

表示z维度的实数向量空间，其重要性权重

的公式推导过程与输入的特征量序列关注度

的计算过程相同，

代表第k个编码器隐藏状态h_k对最终预测的重要性大小。接着解码器将所有的编码器隐藏层状态按照权重求和得到文本向量：Similar to the method in the encoder module, the attention mechanism in the decoder module also requires the hidden layer state output from the previous LSTM unit

and cell state

as an input parameter, where z represents the dimension of the hidden layer in the encoder,

Represents a real vector space in z dimension, its importance weights

The formula derivation process of and the attention of the input feature sequence

The calculation process is the same,

Represents the importance of the kth encoder hidden state h _k to the final prediction. Then the decoder sums all the encoder hidden layer states according to the weights to obtain the text vector:

Combining the historical load data {L _TP ,L _T-P+1 ,...,L _T-1 } and the obtained text vector, after vector splicing and linear change, the adaptively extracted decoding layer input can be obtained:

为t-1时刻的负载与计算得到的文本向量的拼接，m是前面所述的解码器中的隐藏层维度，

表示m+1维度的实数向量空间。

和

是线性变换过程中待学习的参数。接着利用计算得到的

更新解码器隐藏层状态：in,

is the splicing of the load at time t-1 and the calculated text vector, m is the hidden layer dimension in the decoder described above,

Represents a real vector space of dimension m+1.

and

is the parameter to be learned in the linear transformation process. Then use the calculated

Update the decoder hidden layer state:

Among them, f ₂ is a nonlinear activation function of an LSTM unit, and its specific update calculation method is consistent with f ₁ .

The output layer is composed of a multi-layer perceptron. The final hidden layer state output by the encoding and decoding model, namely {d _TP , d _T-P+1 ,..., d _T-1 }, is used as input, and the three-layer perceptron network is used as input. The output is the final model prediction result, where PReLU is used as the activation function in the first two layers of the multilayer perceptron:

f ₃ =max(μd _t ,d _t )

where _f3 represents the PReLU activation function and μ is a parameter that is only updated during training. The PReLU activation function avoids the problem that some parameters cannot be updated. The activation function of the last layer of perceptron is a sigmoid function to ensure that the prediction results can be limited within a reasonable range.

Finally, after T time steps, the nonlinear part of the final load prediction result is obtained

代表在经历了T个时间步后对于负载预测的非线性部分，我们用 T表示时间步大小，non表示上述非线性的标识，F_non用于表示上述整个负载预测的非线性计算过程，{L_T-P,...,L_T-1}表示T时间步之前P-1个时间步的负载大小， {X_T-P,...,X_T}表示T时间步前P个时间步的输入特征序列。

是隐藏层状态和文本向量的拼接向量，

表示z+m维度的实数向量空间，参数W_y和b_w实现了将拼接向量映射为解码层隐藏层状态的尺寸，其中W_y表示拼接向量在解码器计算过程中对应的非线性映射权重，b_w表示偏置值，其计算实现过程由计算机程序实现，同理

和u_w分别代表输出层中的非线性映射权重和偏置值，使用F_non代表上述过程的神经网络预测函数，

表示计算结果即最终预测结果的非线性组成部分。in,

Represents the nonlinear part of load prediction after T time steps, we use T to represent the time step size, non to represent the above nonlinear identifier, F _non to represent the nonlinear calculation process of the entire load prediction above, {L _TP ,...,L _T-1 } represents the load size of P-1 time steps before T time step, {X _TP ,...,X _T } represents the input feature sequence of P time steps before T time step .

is the concatenation vector of the hidden layer state and the text vector,

Represents the real vector space of z+m dimension, and the parameters W _y and b _w realize the size of mapping the splicing vector to the hidden layer state of the decoding layer, where W _y represents the corresponding nonlinear mapping weight of the splicing vector in the decoder calculation process, b _w represents the bias value, and its calculation and realization process is realized by a computer program.

and u _w represent the nonlinear mapping weight and bias value in the output layer, respectively, and use F _non to represent the neural network prediction function of the above process,

Represents the nonlinear component of the calculation result, which is the final prediction result.

In the prediction of the linear change of the load, the load data of the past multiple time steps are used as the input of the autoregressive model according to the collected historical load data to predict the load prediction value of the next time step. In this way, the long-term linear change trend of load changes can be explored, and the problem of insensitivity of the input and output scales of the neural network model can be avoided.

The specific calculation formula of the autoregressive model is:

表示计算结果即最终预测结果的线性组成部分。Among them, {L _TP ,L _T-P+1 ,...,L _T-1 } is the historical data, ε _T is the random disturbance variable, and λ _t is the weight corresponding to each moment. In the design of the autoregressive model, initialization and automatic numerical update are realized, and F _linear is used to represent the autoregressive prediction function of the above process.

Represents the linear component of the calculation result, which is the final prediction result.

In step 3, the obtained load prediction result is returned to the data center resource allocator, and the resource allocator sends the generated resource allocation strategy to the cloud data center for service resource allocation.

As shown in Figure 1 and Figure 5, the load prediction result is applied to the resource allocator, specifically:

和

作为最终的预测结果。In the above analysis and modeling process, as shown in the real load change trend diagram shown in Figure 6, the load change process is often the coexistence of an overall linear change trend and a nonlinear change trend. Therefore, in the load prediction analysis, comprehensive analysis of the nonlinear and linear components of the load helps to improve the accuracy of the prediction results. Therefore, the proposed method combines the nonlinear prediction component and linear prediction component of the load, that is, the output results of the neural network model and the autoregressive model.

and

as the final prediction result.

Its final prediction result can be expressed as:

表示在T时刻得到的自回归模型的预测分量，

表示在T时刻神经网络模型的预测分量。[d_T；C_T]∈R^p+m是隐藏层状态和文本向量的拼接向量，参数W_y和b_w实现了将拼接向量映射为解码层隐藏层状态的尺寸，

和b_v分别代表输出层中的非线性映射权重和偏置值，{L_T-P,L_T-P+1,...,L_T-1}代表历史负载数据， {X_T-P,X_T-P+1,...,X_T}代表输入的序列向量。in,

represents the predicted component of the autoregressive model obtained at time T,

represents the prediction component of the neural network model at time T. [d _T ; C _T ]∈R ^p+m is the splicing vector of the hidden layer state and the text vector, and the parameters W _y and b _w realize the size of mapping the splicing vector to the hidden layer state of the decoding layer,

and b _v represent the nonlinear mapping weights and bias values in the output layer, respectively, {L _TP ,L _T-P+1 ,...,L _T-1 } represent historical load data, {X _TP ,X _{T- P+1} ,...,X _T } represents the input sequence vector.

Next, the resource allocator generates a corresponding resource scheduling policy according to the prediction result, and flexibly adjusts the occupied resources of each server virtual machine.

反馈给资源分配器，资源分配器根据负载预测结果产生相应的资源分配策略，云端数据中心根据该策略动态调整各虚拟机的资源分配情况。The system needs to predict the virtual machine load on each server at time T

Feedback to the resource allocator, the resource allocator generates a corresponding resource allocation strategy according to the load prediction result, and the cloud data center dynamically adjusts the resource allocation of each virtual machine according to the strategy.

与云端数据中心收集到的当前实际负载数据反馈给预测模型，从而模型可以不断获得新的实验数据进行模型训练，从而进一步提升模型预测的性能，减少误差。Step 4: Calculate the load prediction results of multiple times in the past

The current actual load data collected by the cloud data center is fed back to the prediction model, so that the model can continuously obtain new experimental data for model training, thereby further improving the performance of model prediction and reducing errors.

Through the system, the load prediction result and the actual load error can be continuously fed back to the model, thereby reducing the prediction deviation of the load model due to lack of data.

The proposed resource allocator generates the corresponding resource allocation results after predicting the future load of the server cluster in the cloud multi-data center environment based on the load forecasting method. The specific implementation details of the resource allocator can be realized by computer code, and the scheduling method is selected as follows: according to the load prediction result, allocate more computing resources such as the number of CPU cores, memory, etc. to the server that may bear more computing tasks in the future. , so that the current server can continue to support its service operation. When the computing resources are allocated to a single server, the resource allocator should consider redistributing the upcoming computing tasks to avoid task blocking or even system collapse due to excessive task load on a single server node. If the load is balanced among the servers, a counter can be set to distribute the computing tasks evenly. In a multi-data center environment, when the amount of computing resources is limited, you should consider returning computing tasks to the cloud for reallocation to avoid computing delays caused by the accumulation of tasks.

To sum up, the present invention predicts the future load changes of each virtual machine on the server according to the collected log record files, and utilizes the resource allocator to realize dynamic adjustment and configuration of data center resources. Considering the long-term trend of load sequence changes and the interdependence of load characteristics and other characteristic sequences in the data center environment, a fusion model based on neural network and autoregressive method is designed to predict the load of multiple data centers. The robustness of the prediction results is improved, and the scale insensitivity problem of the neural network model is avoided. And the flexibility of the model is guaranteed, so that it can adapt to the data center load forecast with different changing trends.

Example 2

Based on the same inventive concept as Embodiment 1, an embodiment of the present invention is a load prediction device oriented to multiple data centers in the cloud, including:

The data processing module is used to obtain the log record file that records the resource usage of the virtual machine at each time point, extract the required feature data and historical load data from it, and convert the feature data and historical load data into corresponding input features sequence and historical load vectors;

The nonlinear component prediction module is used to calculate the nonlinear component of load prediction by using the pre-built neural network model based on the obtained input feature sequence and historical load vector;

The linear component prediction module is used to calculate the linear component of the load prediction by using the pre-built autoregressive model based on the obtained historical load vector;

The forecasting result calculation module integrates the nonlinear and linear components of load forecasting to obtain the final load forecasting result.

For the specific implementation scheme of each module of the device of the present invention, refer to the implementation process of each step of the method in Example 1.

Example 3

Based on the same inventive concept as Embodiment 1, a resource allocator in this embodiment of the present invention includes:

A load prediction module, configured to calculate and obtain the load prediction result of the virtual machines on each server in the cluster in the cloud multi-data center environment based on the above method;

The resource scheduling module is used for generating corresponding resource scheduling policies based on the load prediction results of the virtual machines on each server.

For the specific implementation scheme of each module of the device of the present invention, refer to the implementation process of each step of the method in Example 1.

As will be appreciated by those skilled in the art, the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

The above are only the preferred embodiments of the present invention. It should be pointed out that for those skilled in the art, without departing from the technical principles of the present invention, several improvements and modifications can also be made. These improvements and modifications It should also be regarded as the protection scope of the present invention.

Claims (7)

1. A load prediction method oriented to multiple data centers in the cloud, characterized in that it comprises the following processes: Obtain the log record file that records the resource usage of the virtual machine at each point in time, extract the required feature data and historical load data from it, and convert the feature data and historical load data into the corresponding input feature sequence and historical load vector; Based on the obtained input feature sequence and historical load vector, the nonlinear component of load prediction is calculated by using a pre-built neural network model; Based on the obtained historical load vector, the linear component of load prediction is calculated by using a pre-built autoregressive model; Integrate the nonlinear and linear components of load prediction to get the final load prediction result; The neural network model includes: an encoder, a decoder and a multi-layer perceptron network; wherein, the input of the encoder is the collected input feature sequence, and the input of the decoder is the adaptively extracted input feature sequence output by the upper-layer encoder , the input of the multilayer perceptron network is the text vector output by the decoder; Described based on the obtained input feature sequence and historical load vector, using a pre-built neural network model to calculate the nonlinear component of load prediction, including: The encoder module includes an input attention layer, a softmax layer, and an LSTM neural network layer. The output of each layer is the input of the next layer; the hidden layer state output by the previous LSTM unit is updated in the encoder module data loop update.

and the cell state s _t-1 as input parameters, use: h _t =f ₁ (h _t-1 ,X _t ) Represents an update calculation process; where f ₁ represents the LSTM unit,

Represents a real vector space of m dimension, X _t represents the input feature sequence at time t; In an LSTM cell, at each time step there are the following computations:

Among them, f _t , i _t , o _t represent forgetting gate, input gate and output gate respectively;

are the weight matrices applied to r _t and h _t-1 , respectively,

represents a real vector space of 4m × d _r dimensions,

Represents a real vector space of 4m×m dimensions, m is the hidden layer dimension, d _r is the vector dimension of the input r _t ; r _t is the input at time t; h _t-1 is the hidden state vector output at time t-1;

is the candidate unit state vector output by the neural network model at the current moment; sigmoid and tanh represent different activation functions respectively; Then, the weight corresponding to each input feature sequence is obtained through the input attention mechanism

in,

is an intermediate variable with no specific actual meaning,

and

is the parameter that needs to be learned in the attention mechanism model,

represents a real vector space of dimension T,

represents a real vector space of dimension T × 2m,

Represents the real vector space of T×T dimension; tanh represents the hyperbolic tangent activation function, and exp represents the exponential function;

The calculation is processed through the softmax layer; Through the attention weight, the adaptively extracted input feature sequence is obtained:

use

Represents the adaptive input feature sequence at time t, and then updates the hidden layer state of the LSTM unit as:

In the decoder module, similar to the encoder module, the attention mechanism in the decoder module combines the hidden layer states output by the previous LSTM unit

and cell state

as an input parameter, where z represents the dimension of the hidden layer in the encoder,

Represents a real vector space in z dimension, its importance weights

The formula derivation process of and the attention of the input feature sequence

The calculation process is the same,

Represents the importance of the kth encoder hidden state h _k to the final prediction; then the decoder sums all the encoder hidden layer states according to the weight to obtain the text vector:

Combining the historical load data {L _TP ,L _T-P+1 ,...,L _T-1 } and the obtained text vector, through vector splicing and linear change, the adaptively extracted decoding layer input is obtained:

in,

is the splicing of the load at time t-1 and the calculated text vector, m is the hidden layer dimension in the decoder,

Represents a real vector space of dimension m+1;

and

is the parameter to be learned in the linear transformation process; then use the calculated

Update the decoder hidden layer state:

Among them, f ₂ is a nonlinear activation function of an LSTM unit, and its specific update calculation method is consistent with f ₁ ; The output layer takes the final hidden layer state output by the encoder/decoder model, namely {d _TP ,d _T-P+1 ,...,d _T-1 } as input, and outputs the final model prediction result through the three-layer perceptron network output. , where PReLU is used as the activation function in the first two layers of the multilayer perceptron: f ₃ =max(μd _t ,d _t ) where f ₃ represents the PReLU activation function, μ is a parameter that is only updated during the training process; the activation function of the last layer of perceptrons is the sigmoid function; The nonlinear part of the final load prediction result is obtained after T time steps

in,

Represents the nonlinear part of the load forecast after T time steps, T represents the time step size, non represents the nonlinear identifier, F _non is used to represent the nonlinear calculation process of the entire load forecast, {L _TP , .. .,L _T-1 } represents the load size of P-1 time steps before T time step, {X _TP ,...,X _T } represents the input feature sequence of P time steps before T time step;

is the concatenation vector of the hidden layer state and the text vector,

Represents the real vector space of z dimension, the parameters W _y and b _w realize the size of mapping the splicing vector to the hidden layer state of the decoding layer, where W _y represents the nonlinear mapping weight of the splicing vector in the decoder calculation process, b _w Indicates the bias value, the same is true

and u _w represent the nonlinear mapping weights and bias values in the output layer, respectively, and use F _non to represent the neural network prediction function of the above process. 2. The load prediction method oriented to multiple data centers in the cloud according to claim 1, wherein the characteristic quantity comprises: sampling and recording time, configuration of the number of virtual machine cores, CPU capacity, and memory configuration capacity of the virtual machine , the effective usage of virtual machine memory, disk read throughput, disk write throughput, network reception throughput, and network transmission throughput; the load refers to the effective CPU usage. 3. The load prediction method oriented to multiple data centers in the cloud according to claim 1, wherein the converting the feature quantity data into a corresponding input feature sequence comprises: The feature quantity data is segmented by sliding window to form a time series with a fixed time step as the input feature sequence. 4. a kind of load prediction method oriented to cloud multi-data center according to claim 1, is characterized in that, the concrete calculation formula of described autoregressive model is:

Among them, {L _TP ,L _T-P+1 ,...,L _T-1 } is the historical data, ε _T is the random disturbance variable, λ _t is the weight corresponding to each moment, and F _linear represents the autoregressive prediction function,

Represents the linear component of the calculation result, which is the final prediction result. 5. An apparatus based on the method for load prediction for multiple data centers in the cloud according to any one of claims 1-4, characterized in that, comprising: The data processing module is used to obtain the log record file that records the resource usage of the virtual machine at each time point, extract the required feature data and historical load data from it, and convert the feature data and historical load data into corresponding input features sequence and history load vectors; The nonlinear component prediction module is used to calculate the nonlinear component of load prediction by using the pre-built neural network model based on the obtained input feature sequence and historical load vector; The linear component prediction module is used to calculate the linear component of the load prediction by using the pre-built autoregressive model based on the obtained historical load vector; The forecasting result calculation module integrates the nonlinear and linear components of load forecasting to obtain the final load forecasting result. 6. A resource scheduling method for multiple data centers in the cloud, characterized in that it comprises the following processes: Calculate the load prediction result of the virtual machine on each server in the cluster in the cloud multi-data center environment based on the method of any one of claims 1-4; A corresponding resource scheduling policy is generated based on the load prediction results of the virtual machines on each server. 7. A resource allocator, characterized in that it comprises: A load prediction module, configured to calculate and obtain the load prediction result of the virtual machines on each server in the cluster in the cloud multi-data center environment based on the method of any one of claims 1-4; The resource scheduling module is used for generating corresponding resource scheduling policies based on the load prediction results of the virtual machines on each server.

Images