CN108170529A - A kind of cloud data center load predicting method based on shot and long term memory network - Google Patents

A kind of cloud data center load predicting method based on shot and long term memory network Download PDF

Info

Publication number
CN108170529A
CN108170529A CN201711433325.9A CN201711433325A CN108170529A CN 108170529 A CN108170529 A CN 108170529A CN 201711433325 A CN201711433325 A CN 201711433325A CN 108170529 A CN108170529 A CN 108170529A
Authority
CN
China
Prior art keywords
sample
shot
training
long term
lstm
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201711433325.9A
Other languages
Chinese (zh)
Inventor
毕敬
许伯睿
乔俊飞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing University of Technology
Original Assignee
Beijing University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing University of Technology filed Critical Beijing University of Technology
Priority to CN201711433325.9A priority Critical patent/CN108170529A/en
Publication of CN108170529A publication Critical patent/CN108170529A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The present invention discloses a kind of cloud data center load predicting method for being based on shot and long term memory network (LSTM), it is intended to solve the problems, such as that the limited computing resource of cloud data center can not obtain optimal utilization.This method makes training sample and test sample based on the magnanimity historical record of cloud data center, and the neural network that another structure one is formed by connecting by LSTM units, constantly bulk input training sample, obtains output valve；The optimum algorithm of multi-layer neural network uses newer adaptability moments estimation method, constantly updates the parameter in each unit by repetitive exercise so that the overall situation is optimal；After training, test sample need to only be inputted to network with regard to next predicted value of the sample sequence can be obtained；If constantly update list entries with predicted value, additionally it is possible to obtain the prediction value sequence of following a period of time.

Description

A kind of cloud data center load predicting method based on shot and long term memory network
Technical field
Field of cloud computer technology of the present invention more particularly to a kind of cloud data center load based on shot and long term memory network are pre- Survey method.
Background technology
Cloud computing is the increase, use and delivery mode of the related service based on internet, is usually directed to and passes through internet Dynamic easily extension and the often resource of virtualization be provided, it can be by network to magnanimity, the user of different priority levels On-demand computing resource and result of calculation are provided.The pattern that a kind of pay-for-use of resource generally use in cloud data center uses Dynamically provide a user service.
According to USA National Institute of Standard and Technology (National Institute of Standards and Technology, NIST) definition, this pattern provides network access available, easily, on-demand, into configurable Computing resources shared pool (resource includes network, server, storage, application software, service), and these resources can be quick It provides.But it is also not enough, because the demand that information-intensive society calculates large-scale data is huge, and still that these resources often again more There is the gesture of continuous growth, often will appear the situation of extensive task requests " swaring forward ".In this case, in cloud data The heart often causes calculating speed to slow down because failing computing resource doing optimum allocation, and working efficiency is low, large quantities of long-term places of request In wait state, the abnormal conditions such as the energy wastes.This not only results in calculating service profit and reduces, and input-output ratio declines, also Prestige and public praise can be damaged.To ensure that cloud data center works normally always, the efficient and whole calculating tasks in ground of guaranteeing both quality and quantity are appointed Business scheduling is essential.It is effectively dispatching premise is that estimating the task load of each priority in following a period of time in advance With unit task average resource applications, the Accurate Prediction to this two indexs is correct allotment computing resource, complete so as to reach The powerful guarantee of office's optimization.
Shot and long term memory-type network (Long Short-Term Memory, LSTM), is time recurrent neural network (RNN) A kind of special shape.Different from feedforward network, it is constantly using the output of itself as input, with the increase of recurrence number, Network can influence the judgement of t-th of time step in the judgement of the t-1 time step, this feedback cycle in actual life " by It is known to push away it is found that pushing away again unknown " derivation behavior be consistent in itself, this method has memory.
For " distance learning energy force difference " existing for traditional RNN (due to there are gradient disappearance, when learning information and prediction bits Interval widen, it is apparent that RNN infers that ability declines) problem, LSTM improves each neural unit, devised cellular State C and increase, " door " structure of removal information to cell state ability, internal structure are as shown in Figure 2.One door is by one A sigmoid layers of h and pointwise multiplication operation composition, the purpose is to screen letter with the weight of sigmod layers of generation Breath, control data flowing, determines whether information passes through.Door there are three being gathered around in one LSTM unit：It is that " forgetting door " (determines respectively Abandon which of C information), " input gate " (determining what information is inserted into C) and " out gate " (determines what is exported Value).By Fig. 2 it is evident that：In t moment, LSTM units have sample Xt, upper unit output ht-1, a upper unit it is cellular State Ct-1Three inputs, these inputs form new cell state C by the processing of doortH is exported with new unittAnd it flows to Next unit, several such units, which join end to end, is formed shot and long term memory network.As training sample continually enters, net Network constantly learns, and extracts its rule and feature, and the weight of all doors also can constantly be adjusted by optimization method, is finally reached Global optimum realizes Accurate Prediction.
Continuous development and burning hot, more and more Development Frameworks appearances, for learner and developer with artificial intelligence The Tensorflow that provides convenience be Google release the open source software library for numerical calculation, it by user design calculation Method is described with a width data flow diagram.Multidimensional data is flowed and can dynamically be adjusted as " tensor " (tensor), especially suitable Close the structure of neural network, training and the applications such as classification, reasoning.Tensorflow provides abundant call method, can be automatic It differentiates and derivative, the calculating details that complexity need not be write by making user can build computation model；When program starts to perform, Node in model can be automatically assigned in the equipment such as CPU, GPU by Tensorflow, realized process optimization, sent out to greatest extent Wave the calculating potential of equipment；It is also equipped with portability, can trained model be moved on to mobile phone without changing code, serviced It is run in device or other clusters, the features such as user group is huge, is one of current most popular artificial intelligence development platform.
In summary several sections of introductions and analysis to the relevant technologies are pair in the load estimation question essence of cloud data center The prediction of time series (task requests amount sequence and unit request resource bid amount sequence).In view of cloud data center presently, there are Resource can not optimal assignment problem, need propose one kind using LSTM neural networks as model, by by a large amount of history data sets Into each priority level task requests amount sequence and unit request resource bid amount sequence train and complete optimization, it is accurate pre- This two refer to calibration method in survey following a period of time.
Invention content
The purpose of the present invention is to provide a kind of operating in cloud data center, the pre- measuring and calculating based on shot and long term memory network Method, the scheduling for computing resource provide sufficient and accurate information support, scheduler module are enable to shift to an earlier date analytic trend and is planned The dispatching method that will be taken well, is ready early, ensures the steady Effec-tive Function of cloud data center.Index to be predicted can be divided into Two classes：Task requests total amount, unit are；Unit asks resource bid amount, no unit.Since every class can be divided into three kinds again, point Three kinds of priority levels of task requests are not corresponded to：It is rudimentary, intermediate and advanced, so a shared 2*3=6 kinds index.
According to an aspect of the invention, there is provided data set production method and a kind of structure of LSTM neural network models Method, including reading data from file；These data are handled, by their " deformations " into the number for meeting Tensorflow standards According to model, the data set being made of time series is made；Initialize the weight matrix of LSTM mode inputs layer and output layer, biasing Matrix；The hyper parameter and weight set according to sample batch quantity, the network number of plies, learning rate, time step etc., bias matrix Build LSTM models, acquisition of the Definition Model to data sample, processing, the way of output.
According to another aspect of the present invention, provide it is a kind of data sample input LSTM network models, allow its learn sequence Row rule adjusts each door weight to optimize global training method and iterative rolling forecast method, including with defeated Training sample (being referred to as " tensor " in Tensorflow frames, this statement hereafter will be used often) the counting loss letter entered Number；Make independent variable adaptability moments estimation (Adaptive moment estimation, Adam) algorithm with loss function to carry out Global optimization；Repetitive exercise repeats above two steps and periodically preserves the model adjusted and and then using test sample to be defeated Enter, the predicted value of following n period index is exported with " final version " network model iteration.
In conclusion a kind of cloud data center load predicting method based on shot and long term memory network, includes the following steps：
S1, with storage data creating historical time sequence hereof and data set；
S2, structure shot and long term neural network model；
S3, training LSTM networks：Training sample, counting loss letter are iteratively imported into shot and long term neural network model Number, and global optimization is carried out based on this, the feature of training sample and the relationship of numerical value and sequential are constantly extracted, until repeatedly In generation, terminates；
S4, after the training stage, to shot and long term neural network model；Middle importing test sample, the following number of iteration output The numerical value of a time step index to be predicted forms predicted value time series.
Preferably, data set is divided into two parts in step 1：
First part is " tensor subsets ", is replaced with " X subsets "：The tensor be a time series, be training or The minimum unit of LSTM networks is inputted during test, length is num_step, and construction method is that iteratively taking-up connects from matrix Continuous data, are converted to list and insert：
Second part is " sub-set of tags ", is replaced, is made of the corresponding labels of each tensor, Y subsets are every with " Y subsets " Sequential value on a position is the subsequent value of X subset corresponding position sequential values：
Preferably, step 2 is specially：
Step 2.1, each this input of lot sample LSTM networks, each unit can produce after screening and handling sample in network Raw two values are simultaneously passed to next unit：One is referred to as " cell state ", abbreviation C, the other is " unit output ", abbreviation h； LSTM networks have num_unit unit, have batch_size sample sequence in every batch of sample, therefore a collection of sample can be produced successively Raw num_unit*batch_size C value and h values, i.e.,：
Step 2.2 removes all c value sequences, retains all h value sequences, generates a new matrix；
This new matrix is multiplied by step 2.3 with output layer weight matrix w_out, then with output layer bias matrix b_out It is added, obtains pred.
Preferably, step 3 is specially：
1) first batch_size sample is taken out from X, two subsets of Y, is respectively designated as x and y；
2) x is inputted into LSTM models, acquires pred matrixes；
3) counting loss amount tensor loss.
4) using loss amount loss and learning rate lr as parameter, adaptability moments estimation (Adam) optimizer, meter are initialized The gradient of loss is calculated, then gradient is applied on variable, updates the weight of door in all units of LSTM networks, returns to one A tensor that result is exported comprising training operation；
5) batch_size sample of next group is taken out from X, Y subsets, updates x and y；
6) such as x, y is not sky, then repeatedly step 2- steps 5；Otherwise start next iteration.
Preferably, in step 3, in the LSTM network training stages, after giving learning rate lr and calculating loss amount loss, It is adjusted using adaptability moments estimation (Adaptive moment estimation) method, updates and forget in LSTM network models The weight of door, input gate and out gate.The present invention uses novel Neural network optimization：Adam algorithms substitution it is traditional with Machine gradient descent method.It by calculate single order moments estimation and the second order moments estimation of gradient and for different parameter designing independence from Adaptability learning rate, is suitble to solve with large-scale data and parameter, and the problem of high is required to pace of learning.It is calculated with other optimizations Method, if stochastic gradient descent (SGD) method, momentum method are compared with AdaGrad methods, faster, learning effect becomes apparent from its convergence rate, Learning rate disappearance can be also corrected, is absorbed in local optimum, loss function fluctuates the problems such as big.Because to the prediction of H to the need of speed Accuracy rate of summing is very high, so Adam algorithms are most preferred.
Adam algorithms are related to following constant：α is step factor；β1It is single order moments estimation attenuation rate；β2It is that second moment is estimated Count attenuation rate；ε is very small, close to zero number.In Tensorflow frames, the default setting difference of this four constants It is 0.001,0.9,0.999,10E-8.
If random targets function is f (θ), mtIt is the first moment vector of t moment parameter θ；vtIt is the second moment of t moment parameter θ Vector；Initial value is all zero.Algorithm is that iteration carries out, and each iteration is all along with the update of θ.When θ does not restrain, cycle changes It is operated below substitute performance.It is specific as follows：
1) time step t adds 1, t=t+1.
2) gradient of the t moment object function to parameter θ is obtained,
3) update first order and second order moments estimation, mt←β1·mt-1+(1-β1)·gt； vt←β2·vt-1+(1-β2)· gt 2。
4) the first order and second order moments estimation after drift correction is calculated,
5) undated parameter vector,
This iterative process is until θtUntil convergence.Wherein, mtAnd vtCan respectively it regard as to gtAbsolute value it is expected and gtSquare The estimation of absolute value.In Tensorflow, it is known that learning rate just builds optimizer using Adam algorithms, trains each door The optimal weight of structure.
Beneficial effects of the present invention are as follows
Technical solution of the present invention can solve in cloud data center it is of all categories request reach irregularities and unit The problem that the otherness of request resource bid is brought to request scheduling so that cloud data center " can provide for a rainy day ", in advance very Later variation tendency is estimated for a long time, and then determines resource allocation mode and dispatching algorithm so that cloud data center is always It can be operated with state optimization, the most efficient mode of the utilization of resources.By the program and other similar scheme lateral comparisons, training institute It takes time shorter, learning efficiency higher, moreover it is possible to evade common local optimum, the defects of gradient disappears, and convergence rate is slow, these It is all of practical significance very much to the cloud data center for extremely emphasizing efficiency.
Description of the drawings
The specific embodiment of the present invention is described in further detail below in conjunction with the accompanying drawings：
Fig. 1 is the flow chart of the cloud data center load predicting method based on shot and long term memory network；
Fig. 2 is the internal structure chart of shot and long term memory network basic unit；
Fig. 3 is the complete training flow of LSTM networks；
Fig. 4 is the flow that prediction value sequence is obtained with test sample.
Specific embodiment
The implementing procedure and points for attention of the present invention are further elaborated below.As it was noted above, in cloud data center Index to be predicted shares six kinds, but most contents are suitable for predicting this six kinds of indexs in algorithm.If a certain step is to being directed to Different types of premeasuring has different treating methods, has special instruction.Algorithm is write with python language, imports Tensorflow, data analysis packet pandas, numerical computations expanding packet numpy and for drawing image matplotlib.pyplot.In this part, some index to be predicted is referred to always with " H ", the prediction of remaining five kinds of index Method is substantially the same therewith.
S1, with storage data creating historical time sequence hereof and data set；
Historical data is frequently stored in the file of csv forms.To predict H, the first step is exactly the reading H from file Historical data forms a time series.The total data of six kinds of indexs is by calling the method in pandas to obtain in csv files , the historical time sequence of H is created with numpy, and parameter is whole historical datas of H.
Start the making of data set below.Since the historical data of these magnanimity can be there are numerical value polarization, dimension is not With situations such as even gap is huge, and these will cause learning process to restrain not restrain even slowly, in order to which data is made more " to put down It " and under identical dimension analyzes, needs that initial data is normalized.First kind index (task requests amount) value Distribution it is more random, there are many influence factor, not necessarily approximate Gaussian distribution, therefore logarithm normalization method is used to it：Sample data =denary logarithm is taken to initial data；The distribution of the second class index (unit task average resource applications) value is approximate Gaussian Profile uses it standard deviation normalization method to be suitble to the most：Sample data=(initial data-data grand mean)/data are total Standard deviation.The subsequent step of algorithm will often use matrix calculating, be exactly to carry out increasing dimension to the sample sequence of H in next step, become Shape is the two-dimensional matrix of [n, 1]：
[[t0] [t1] ... [tn-1]]T。
To achieve the purpose that " in training optimizing ", data set is needed to be divided into two parts：First part is " tensor Collection " (is replaced) with " X subsets "：In this algorithm, the essence of tensor is also a time series, is inputted when being training or test The minimum unit of LSTM networks, length are num_step, and construction method is that continuous data is iteratively taken out from matrix, conversion Into list and insert：
Second part is " sub-set of tags " (being replaced with " Y subsets "), is made of the corresponding labels of each tensor.Y subsets Sequential value on each position is the subsequent value of X subset corresponding position sequential values：
S2, structure shot and long term neural network model, can be correct, effectively processing input and calculating output；
The partial parameters of LSTM models need pre-set value, they are referred to as hyper parameter.This algorithm is related to super Parameter has eight, is learning rate lr, input layer dimension input_size, output layer dimension output_size, single batch respectively Number of training batch_size, individual layer LSTM unit numbers num_unit, time step number num_step, repetitive exercise number Epochs and test sample number n_train.Since the prediction category single factor test to H future values is predicted, so LSTM networks The number of plies is 1, input_size and output_size is 1.Training sample is in large scale, and tensor is often thousands of.It is single Although the secondary sample for being input into network excessively can reduce iterations, convergence rate is slow, is easily trapped into local optimum；It is input into very few Although sample can accelerate convergence rate and improve precision, too small batch_size can not embody the advantage of parallel computation and Training stability is poor.So batch_size should not be too large or too small, the sample set of small batch is optimal.
In the usual course, the weight of each LSTM basic units input layer and output layer be set as meeting Gaussian Profile with Machine number, amount of bias are set as constant, and such as 0.1.Input layer weight matrix w_input, output layer weight matrix w_output, input Layer bias matrix b_input and output layer bias matrix b_output can be built according to this.
The construction work of model is encapsulated in a function, it is therefore an objective to describe a LSTM network using Tensorflow Abstract data flow diagram provides " raw material " for subsequent step.Due to the single batch sample needed for training stage and test phase Number is different, so the parameter of function is set as batch, if batch=batch_size, illustrates the network for training；If Batch=1 illustrates the network for testing (one tensor sample of test input every time).
Realize that this algorithm need to only use basic LSTM models, so it is basic to enable each network element BasicLSTMCell.Such cell mono- shares num_unit, without advanced variants such as clipping, peep-hole.Net Network is individual layer, therefore does not introduce the DropoutWrapper that multilayer LSTM is usually used in preventing over-fitting.
According to the regulation of Tensorflow handbooks, network model needs when handling a collection of sample first will be continuous from X subsets It takes out, into one " vertical bar ", (shape is [batch_size* to the matrix " elongation " being made of batch_size sample sequence Num_step, 1]), composing training sample matrix train_matrix.
Training sample matrix cannot come into LSTM networks directly as input, need same input layer weight matrix w_input Multiplication is added again with input layer bias matrix b_input is converted to shape as [batch_size, num_step, num_unit] Matrix r eal_input after can really be inputted as network, and be passed to tensorflow's as parameter (purpose for calling tf.nn.dynamic_rnn methods is instructed with given input and initial model to tf.nn.dynamic_rnn methods Practice class RNN networks and return to training result).
Real_input=train_matrix* [w0w1...wnum_unit-1]+[b0b1..bnum_unit-1]T
The final output of S2 is [batch_size, a 1] shape matrix (being referred to below with " pred "), by LSTM networks " predicted value of lower a moment " obtained after calculating the sample sequence of all inputs in the lot sample sheet is formed.Because in training shape Every batch of sample has batch_size sequence under state, therefore has batch_size element in pred.It is the calculating step of pred below Suddenly：
1) tf.nn.dynamic_rnn, is run, takes out second return item [batch_size, 2*num_unit] type square Battle array, when inputting network by each sample sequence in the lot sample sheet, c values and h values that all LSTM units generate form, knot Structure is shown below：
2), remove all c value sequences, retain all h value sequences, generate a new matrix；
3), this new matrix is multiplied with output layer weight matrix w_out, then with output layer bias matrix b_out phases Add, obtain pred.
Training to LSTM networks is a cyclic process, and cycle can all traverse entire training sample set every time, and training is every During a collection of sample will counting loss amount, then do a suboptimization with Adam algorithms.Cycle-index is represented with variable epoch, is somebody's turn to do Value is adjustable, and when having performed epoch cycle, network model just trains.
S3, training LSTM networks：Iteratively import training sample into this model, counting loss function, and as Basis carries out global optimization, constantly extracts the feature of training sample and the relationship of numerical value and sequential, until iteration terminates；
The network model function designed in S2 is called before training first.Training to LSTM networks is that iteration carries out, altogether Have epoch times, be substantially exactly the following cycle operated：
1) first batch_size sample is taken out from X, two subsets of Y, is respectively designated as x and y；
2) x is inputted into LSTM models, acquires pred matrixes；
3) counting loss amount tensor (being referred to below with " loss ").Forecasting problem is substantially a regression problem, so Mean Square Error (MSE, mean squared error) should be used.Using Mean Square Error counting loss amount：
Wherein, prediFor " predicted value " of i-th of sample sequence subsequent time of the batch, tagiBe in " sub-set of tags " with Corresponding " label " sequence of the sample sequence (in Tensorflow frames, can be added and subtracted between various sizes of matrix).
4) using loss amount loss and learning rate lr as parameter, adaptability moments estimation (Adam) optimizer, meter are initialized The gradient of loss is calculated, then gradient is applied on variable, updates the weight of door in all units of LSTM networks, returns to one A tensor that result is exported comprising training operation；
5) batch_size sample of next group is taken out from X, Y subsets, updates x and y；
6) such as x, y is not sky, then repeatedly step 2- steps 5；Otherwise start next iteration.
In entire training process, intermediate result is periodically saved as into check point file (binary file, its handle Variable name is mapped to corresponding tensor values, extends entitled .ckpt) it is stored under specified path.The complete training stream of LSTM networks Journey, as shown in Figure 3.
S4, after the training stage, test sample, the following several time step fingers to be predicted of iteration output are imported into model Target numerical value forms predicted value time series.
Into after forecast period, input LSTM networks are the test samples for being different from training sample, and each sequence is only From in batch.In this way, h value matrixs only there are one, be exactly the sequence subsequent time after w_output and b_output " processing " Predicted value.If wanting to obtain the predicted value at later n moment by cycle tests, should take " step-by-step method "：Every time by upper one Predicted value is connected to behind sequence, is removed leading element and is formed new sequence, is inputted LSTM networks again, is obtained predicted value, so Cycle.If predicted value accuracy rate is not high, test is unsuccessful, can regularized learning algorithm rate, LSTM units number and iterations repeatedly Hyper parameters are waited, later repeatedly step S1-S4, until until the prediction result finally generated is satisfied with.
By repetitive exercise and adjustment, LSTM networks have been adjusted to best, can carry out time series forecasting：First according to path The model of newest preservation in newest .ckpt files is read out, batch_size is set as 1, and input " first survey Try data.It is assumed that predicted value sequence length is also num_step, the forecast of enough " long-range " is not only provided for scheduling of resource in this way, but also Convenient for predicted value with the comparison of actual value.Prediction is rolling and stepping, flow are as shown in Figure 4.
The precise degrees of predicted value can intuitively be obtained by the degree of fitting of two sequence figure lines in plane coordinate system.It is if smart Spend it is not high enough, then constantly adjustment hyper parameter, repeat S1-S4, until two figure line degrees of fitting are satisfactory.
Obviously, above-mentioned implementation process of the invention be only to clearly illustrate example of the present invention, and not be Restriction to embodiments of the present invention.For those of ordinary skill in the art, on the basis of the above description also It can make other variations or changes in different ways.Here all embodiments can not be exhaustive, it is every to belong to this The obvious changes or variations that the technical solution of invention is extended out are still in the row of protection scope of the present invention.

Claims (5)

1. a kind of cloud data center load predicting method based on shot and long term memory network, which is characterized in that this method is included such as Lower step：
S1, with storage data creating historical time sequence hereof and data set；
S2, structure shot and long term neural network model；
S3, training LSTM networks：Iteratively import training sample into shot and long term neural network model, counting loss function, and Global optimization is carried out based on this, constantly extracts the feature of training sample and the relationship of numerical value and sequential, until iteration knot Beam；
S4, after the training stage, to shot and long term neural network model；Middle importing test sample, when iteration output is following several The numerical value of spacer step index to be predicted forms predicted value time series.
2. the cloud data center load predicting method based on shot and long term memory network as described in claim 1, which is characterized in that Data set is divided into two parts in step 1：
First part is " tensor subsets ", is replaced with " X subsets "：The tensor is a time series, is training or test When input LSTM networks minimum unit, length is num_step, and construction method is that consecutive numbers is iteratively taken out from matrix According to being converted to list and insert：
Second part is " sub-set of tags ", is replaced, is made of the corresponding labels of each tensor, each position of Y subsets with " Y subsets " The sequential value put is the subsequent value of X subset corresponding position sequential values：
3. the cloud data center load predicting method based on shot and long term memory network as claimed in claim 2, which is characterized in that Step 2 is specially：
Step 2.1, each this input of lot sample LSTM networks, each unit can generate two after screening and handling sample in network A numerical value is simultaneously passed to next unit：One is referred to as " cell state ", abbreviation C, the other is " unit output ", abbreviation h；LSTM Network has num_unit unit, has batch_size sample sequence in every batch of sample, therefore a collection of sample can be generated successively Num_unit*batch_size C value and h values, i.e.,：
Step 2.2 removes all c value sequences, retains all h value sequences, generates a new matrix；
This new matrix is multiplied by step 2.3 with output layer weight matrix w_out, then with output layer bias matrix b_out phases Add, obtain pred.
4. the cloud data center load predicting method based on shot and long term memory network as claimed in claim 3, which is characterized in that Step 3 is specially：
1) first batch_size sample is taken out from X, two subsets of Y, is respectively designated as x and y；
2) x is inputted into LSTM models, acquires pred matrixes；
3) counting loss amount tensor loss.
4) using loss amount loss and learning rate lr as parameter, adaptability moments estimation (Adam) optimizer is initialized, is calculated The gradient of loss, is then applied to gradient on variable, updates the weight of door in all units of LSTM networks, returns to one The tensor of output result is operated comprising training；
5) batch_size sample of next group is taken out from X, Y subsets, updates x and y；
6) such as x, y is not sky, then repeatedly step 2- steps 5；Otherwise start next iteration.
5. the cloud data center load predicting method based on shot and long term memory network as described in claim 1, which is characterized in that In step 3, in the LSTM network training stages, after giving learning rate lr and calculating loss amount loss, using adaptability moments estimation (Adaptive moment estimation) method adjusts, and updates in LSTM network models and forgets door, input gate and out gate Weight.
CN201711433325.9A 2017-12-26 2017-12-26 A kind of cloud data center load predicting method based on shot and long term memory network Pending CN108170529A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201711433325.9A CN108170529A (en) 2017-12-26 2017-12-26 A kind of cloud data center load predicting method based on shot and long term memory network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201711433325.9A CN108170529A (en) 2017-12-26 2017-12-26 A kind of cloud data center load predicting method based on shot and long term memory network

Publications (1)

Publication Number Publication Date
CN108170529A true CN108170529A (en) 2018-06-15

Family

ID=62521116

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201711433325.9A Pending CN108170529A (en) 2017-12-26 2017-12-26 A kind of cloud data center load predicting method based on shot and long term memory network

Country Status (1)

Country Link
CN (1) CN108170529A (en)

Cited By (38)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108989098A (en) * 2018-08-24 2018-12-11 福建师范大学 A kind of mixing scientific workflow data layout method of the cloud environment towards time delay optimization
CN109063416A (en) * 2018-07-23 2018-12-21 太原理工大学 Gene expression prediction technique based on LSTM Recognition with Recurrent Neural Network
CN109104620A (en) * 2018-07-26 2018-12-28 腾讯科技（深圳）有限公司 A kind of short video recommendation method, device and readable medium
CN109272046A (en) * 2018-09-26 2019-01-25 北京科技大学 Deep learning method based on L2 again regularization Adam switching simulated tempering SGD
CN109299321A (en) * 2018-08-31 2019-02-01 出门问问信息科技有限公司 A kind of song recommended method and device
CN109359624A (en) * 2018-11-14 2019-02-19 浙江农林大学 The prediction technique and prediction meanss of laying hen foraging behaviour neural network based
CN109471698A (en) * 2018-10-19 2019-03-15 中国电子科技集团公司第二十八研究所 System and method for detecting abnormal behavior of virtual machine in cloud environment
CN109522117A (en) * 2018-10-25 2019-03-26 深圳市圆世科技有限责任公司 Data dispatch system on a kind of chain towards under isomerous environment
CN109542585A (en) * 2018-11-14 2019-03-29 山东大学 A kind of Virtual Machine Worker load predicting method for supporting irregular time interval
CN109782392A (en) * 2019-02-27 2019-05-21 中国科学院光电技术研究所 A kind of fiber-optic coupling method based on modified random paralleling gradient descent algorithm
CN109816008A (en) * 2019-01-20 2019-05-28 北京工业大学 A kind of astronomical big data light curve predicting abnormality method based on shot and long term memory network
CN109857459A (en) * 2018-12-27 2019-06-07 中国海洋大学 A kind of E grades of supercomputer ocean model transplants optimization method and system automatically
CN110031214A (en) * 2019-04-09 2019-07-19 重庆大学 Gear hobbing quality online evaluation method based on shot and long term memory network
CN110096349A (en) * 2019-04-10 2019-08-06 山东科技大学 A kind of job scheduling method based on the prediction of clustered node load condition
CN110231976A (en) * 2019-05-20 2019-09-13 西安交通大学 A kind of edge calculations platform container dispositions method and system based on load estimation
CN110297186A (en) * 2019-08-14 2019-10-01 莆田市烛火信息技术有限公司 A kind of new energy car battery parameter detection method
CN110389820A (en) * 2019-06-28 2019-10-29 浙江大学 A kind of private clound method for scheduling task carrying out resources based on v-TGRU model
CN110502432A (en) * 2019-07-23 2019-11-26 平安科技（深圳）有限公司 Intelligent test method, device, equipment and readable storage medium storing program for executing
CN110782016A (en) * 2019-10-25 2020-02-11 北京百度网讯科技有限公司 Method and apparatus for optimizing neural network architecture search
CN111008674A (en) * 2019-12-24 2020-04-14 哈尔滨工程大学 Underwater target detection method based on rapid cycle unit
CN111027591A (en) * 2019-11-13 2020-04-17 西安交通大学 Node fault prediction method for large-scale cluster system
CN111049903A (en) * 2019-12-12 2020-04-21 大连理工大学 Edge network load distribution algorithm based on application perception prediction
CN111143050A (en) * 2018-11-02 2020-05-12 中移（杭州）信息技术有限公司 Container cluster scheduling method and device
CN111179910A (en) * 2019-12-17 2020-05-19 深圳追一科技有限公司 Speed of speech recognition method and apparatus, server, computer readable storage medium
CN111224806A (en) * 2018-11-27 2020-06-02 华为技术有限公司 Resource allocation method and server
CN111415270A (en) * 2020-03-03 2020-07-14 浙江万胜智能科技股份有限公司 Power load intelligent identification method based on L STM learning
CN111491006A (en) * 2020-03-03 2020-08-04 天津大学 Load-aware cloud computing resource elastic distribution system and method
CN111537888A (en) * 2020-05-09 2020-08-14 国网福建省电力有限公司莆田供电公司 Data-driven echelon battery SOC prediction method
CN111787109A (en) * 2020-07-02 2020-10-16 哈尔滨工程大学 Data center load balancing method based on time series prediction
CN111985162A (en) * 2020-08-28 2020-11-24 华中科技大学 Replacement flow shop control method and system based on deep learning
CN112163668A (en) * 2020-09-29 2021-01-01 上海交通大学 Method for reducing time series data transmission based on prediction and cloud edge cooperation
CN112416596A (en) * 2020-12-01 2021-02-26 新华三人工智能科技有限公司 Node scheduling method, device and equipment
CN112532717A (en) * 2020-11-25 2021-03-19 四川易诚智讯科技有限公司 Production process safety monitoring method based on STM32 single chip microcomputer and long-short time memory network
CN113051130A (en) * 2021-03-19 2021-06-29 南京航空航天大学 Mobile cloud load prediction method and system of LSTM network combined with attention mechanism
CN113220450A (en) * 2021-04-29 2021-08-06 南京邮电大学 Load prediction method, resource scheduling method and device for cloud-side multi-data center
CN113220466A (en) * 2021-06-02 2021-08-06 神州数码系统集成服务有限公司 Cloud service load universal prediction method based on long-term and short-term memory model
CN113272825A (en) * 2018-11-21 2021-08-17 亚马逊技术有限公司 Reinforcement learning model training by simulation
CN115102674A (en) * 2022-06-17 2022-09-23 西安电子科技大学 Bi-LSTM network-based high-speed link eye pattern prediction method

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150356075A1 (en) * 2014-06-06 2015-12-10 Google Inc. Generating representations of input sequences using neural networks
CN106502799A (en) * 2016-12-30 2017-03-15 南京大学 A kind of host load prediction method based on long memory network in short-term
CN107239825A (en) * 2016-08-22 2017-10-10 北京深鉴智能科技有限公司 Consider the deep neural network compression method of load balancing
US20170330558A1 (en) * 2013-12-17 2017-11-16 Google Inc. Generating representations of acoustic sequences
CN107481048A (en) * 2017-08-08 2017-12-15 哈尔滨工业大学深圳研究生院 A kind of financial kind price expectation method and system based on mixed model

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170330558A1 (en) * 2013-12-17 2017-11-16 Google Inc. Generating representations of acoustic sequences
US20150356075A1 (en) * 2014-06-06 2015-12-10 Google Inc. Generating representations of input sequences using neural networks
CN107239825A (en) * 2016-08-22 2017-10-10 北京深鉴智能科技有限公司 Consider the deep neural network compression method of load balancing
CN106502799A (en) * 2016-12-30 2017-03-15 南京大学 A kind of host load prediction method based on long memory network in short-term
CN107481048A (en) * 2017-08-08 2017-12-15 哈尔滨工业大学深圳研究生院 A kind of financial kind price expectation method and system based on mixed model

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
ADIT DESHPANDE: "Perform sentiment analysis with LSTMs, using TensorFlow", 《HTTPS://WWW.OREILLY.COM/CONTENT/PERFORM-SENTIMENT-ANALYSIS-WITH-LSTMS-USING-TENSORFLOW/》 *

Cited By (58)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109063416B (en) * 2018-07-23 2019-08-27 太原理工大学 Gene expression prediction technique based on LSTM Recognition with Recurrent Neural Network
CN109063416A (en) * 2018-07-23 2018-12-21 太原理工大学 Gene expression prediction technique based on LSTM Recognition with Recurrent Neural Network
CN109104620A (en) * 2018-07-26 2018-12-28 腾讯科技（深圳）有限公司 A kind of short video recommendation method, device and readable medium
CN109104620B (en) * 2018-07-26 2020-05-19 腾讯科技（深圳）有限公司 Short video recommendation method and device and readable medium
CN108989098A (en) * 2018-08-24 2018-12-11 福建师范大学 A kind of mixing scientific workflow data layout method of the cloud environment towards time delay optimization
CN108989098B (en) * 2018-08-24 2021-06-01 福建师范大学 Time delay optimization-oriented scientific workflow data layout method in hybrid cloud environment
CN109299321A (en) * 2018-08-31 2019-02-01 出门问问信息科技有限公司 A kind of song recommended method and device
CN109299321B (en) * 2018-08-31 2021-07-09 出门问问信息科技有限公司 Method and device for recommending songs
CN109272046A (en) * 2018-09-26 2019-01-25 北京科技大学 Deep learning method based on L2 again regularization Adam switching simulated tempering SGD
CN109471698A (en) * 2018-10-19 2019-03-15 中国电子科技集团公司第二十八研究所 System and method for detecting abnormal behavior of virtual machine in cloud environment
CN109522117A (en) * 2018-10-25 2019-03-26 深圳市圆世科技有限责任公司 Data dispatch system on a kind of chain towards under isomerous environment
CN111143050B (en) * 2018-11-02 2023-09-19 中移（杭州）信息技术有限公司 Method and equipment for dispatching container clusters
CN111143050A (en) * 2018-11-02 2020-05-12 中移（杭州）信息技术有限公司 Container cluster scheduling method and device
CN109359624A (en) * 2018-11-14 2019-02-19 浙江农林大学 The prediction technique and prediction meanss of laying hen foraging behaviour neural network based
CN109542585B (en) * 2018-11-14 2020-06-16 山东大学 Virtual machine workload prediction method supporting irregular time intervals
CN109542585A (en) * 2018-11-14 2019-03-29 山东大学 A kind of Virtual Machine Worker load predicting method for supporting irregular time interval
CN109359624B (en) * 2018-11-14 2021-10-08 浙江农林大学 Neural network-based prediction method and prediction device for feeding behavior of laying hens
CN113272825B (en) * 2018-11-21 2024-02-02 亚马逊技术有限公司 Reinforcement learning model training by simulation
CN113272825A (en) * 2018-11-21 2021-08-17 亚马逊技术有限公司 Reinforcement learning model training by simulation
CN111224806A (en) * 2018-11-27 2020-06-02 华为技术有限公司 Resource allocation method and server
CN109857459B (en) * 2018-12-27 2022-03-08 中国海洋大学 E-level super-calculation ocean mode automatic transplanting optimization method and system
CN109857459A (en) * 2018-12-27 2019-06-07 中国海洋大学 A kind of E grades of supercomputer ocean model transplants optimization method and system automatically
CN109816008A (en) * 2019-01-20 2019-05-28 北京工业大学 A kind of astronomical big data light curve predicting abnormality method based on shot and long term memory network
CN109782392A (en) * 2019-02-27 2019-05-21 中国科学院光电技术研究所 A kind of fiber-optic coupling method based on modified random paralleling gradient descent algorithm
CN110031214A (en) * 2019-04-09 2019-07-19 重庆大学 Gear hobbing quality online evaluation method based on shot and long term memory network
CN110031214B (en) * 2019-04-09 2020-09-22 重庆大学 Hobbing quality online evaluation method based on long-term and short-term memory network
CN110096349A (en) * 2019-04-10 2019-08-06 山东科技大学 A kind of job scheduling method based on the prediction of clustered node load condition
CN110231976A (en) * 2019-05-20 2019-09-13 西安交通大学 A kind of edge calculations platform container dispositions method and system based on load estimation
CN110231976B (en) * 2019-05-20 2021-04-20 西安交通大学 Load prediction-based edge computing platform container deployment method and system
CN110389820A (en) * 2019-06-28 2019-10-29 浙江大学 A kind of private clound method for scheduling task carrying out resources based on v-TGRU model
CN110502432B (en) * 2019-07-23 2023-11-28 平安科技（深圳）有限公司 Intelligent test method, device, equipment and readable storage medium
CN110502432A (en) * 2019-07-23 2019-11-26 平安科技（深圳）有限公司 Intelligent test method, device, equipment and readable storage medium storing program for executing
CN110297186A (en) * 2019-08-14 2019-10-01 莆田市烛火信息技术有限公司 A kind of new energy car battery parameter detection method
CN110782016A (en) * 2019-10-25 2020-02-11 北京百度网讯科技有限公司 Method and apparatus for optimizing neural network architecture search
CN111027591B (en) * 2019-11-13 2022-07-12 西安交通大学 Node fault prediction method for large-scale cluster system
CN111027591A (en) * 2019-11-13 2020-04-17 西安交通大学 Node fault prediction method for large-scale cluster system
CN111049903A (en) * 2019-12-12 2020-04-21 大连理工大学 Edge network load distribution algorithm based on application perception prediction
CN111049903B (en) * 2019-12-12 2021-04-20 大连理工大学 Edge network load distribution algorithm based on application perception prediction
CN111179910A (en) * 2019-12-17 2020-05-19 深圳追一科技有限公司 Speed of speech recognition method and apparatus, server, computer readable storage medium
CN111008674B (en) * 2019-12-24 2022-05-03 哈尔滨工程大学 Underwater target detection method based on rapid cycle unit
CN111008674A (en) * 2019-12-24 2020-04-14 哈尔滨工程大学 Underwater target detection method based on rapid cycle unit
CN111415270A (en) * 2020-03-03 2020-07-14 浙江万胜智能科技股份有限公司 Power load intelligent identification method based on L STM learning
CN111491006A (en) * 2020-03-03 2020-08-04 天津大学 Load-aware cloud computing resource elastic distribution system and method
CN111537888A (en) * 2020-05-09 2020-08-14 国网福建省电力有限公司莆田供电公司 Data-driven echelon battery SOC prediction method
CN111787109A (en) * 2020-07-02 2020-10-16 哈尔滨工程大学 Data center load balancing method based on time series prediction
CN111985162B (en) * 2020-08-28 2024-04-26 华中科技大学 Deep learning-based replacement flow shop control method and system
CN111985162A (en) * 2020-08-28 2020-11-24 华中科技大学 Replacement flow shop control method and system based on deep learning
CN112163668B (en) * 2020-09-29 2023-05-05 上海交通大学 Method for reducing time series data transmission quantity based on prediction and cloud edge cooperation
CN112163668A (en) * 2020-09-29 2021-01-01 上海交通大学 Method for reducing time series data transmission based on prediction and cloud edge cooperation
CN112532717A (en) * 2020-11-25 2021-03-19 四川易诚智讯科技有限公司 Production process safety monitoring method based on STM32 single chip microcomputer and long-short time memory network
CN112416596A (en) * 2020-12-01 2021-02-26 新华三人工智能科技有限公司 Node scheduling method, device and equipment
CN113051130B (en) * 2021-03-19 2023-05-02 南京航空航天大学 Mobile cloud load prediction method and system of LSTM network combined with attention mechanism
CN113051130A (en) * 2021-03-19 2021-06-29 南京航空航天大学 Mobile cloud load prediction method and system of LSTM network combined with attention mechanism
CN113220450B (en) * 2021-04-29 2022-10-21 南京邮电大学 Load prediction method, resource scheduling method and device for cloud-side multi-data center
CN113220450A (en) * 2021-04-29 2021-08-06 南京邮电大学 Load prediction method, resource scheduling method and device for cloud-side multi-data center
CN113220466A (en) * 2021-06-02 2021-08-06 神州数码系统集成服务有限公司 Cloud service load universal prediction method based on long-term and short-term memory model
CN115102674B (en) * 2022-06-17 2023-08-22 西安电子科技大学 Bi-LSTM network-based high-speed link eye diagram prediction method
CN115102674A (en) * 2022-06-17 2022-09-23 西安电子科技大学 Bi-LSTM network-based high-speed link eye pattern prediction method

Similar Documents

Publication Publication Date Title
CN108170529A (en) A kind of cloud data center load predicting method based on shot and long term memory network
Mindermann et al. Prioritized training on points that are learnable, worth learning, and not yet learnt
Hewamalage et al. Recurrent neural networks for time series forecasting: Current status and future directions
Tripathy et al. Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification
Li et al. Prediction for tourism flow based on LSTM neural network
CN110909926A (en) TCN-LSTM-based solar photovoltaic power generation prediction method
Soares et al. An adaptive ensemble of on-line extreme learning machines with variable forgetting factor for dynamic system prediction
Shah et al. Pareto frontier learning with expensive correlated objectives
Sim et al. An expert neural network system for dynamic job shop scheduling
CN111148118A (en) Flow prediction and carrier turn-off method and system based on time sequence
US11366806B2 (en) Automated feature generation for machine learning application
Barman et al. Transfer learning for small dataset
CN110457369A (en) A kind of training method and relevant device of model
Okewu et al. Parameter tuning using adaptive moment estimation in deep learning neural networks
Lu et al. Variance reduced training with stratified sampling for forecasting models
CN111950810A (en) Multivariable time sequence prediction method and device based on self-evolution pre-training
CN115018193A (en) Time series wind energy data prediction method based on LSTM-GA model
Ren et al. A novel solution to jsps based on long short-term memory and policy gradient algorithm
CN108876038B (en) Big data, artificial intelligence and super calculation synergetic material performance prediction method
Ariafar et al. Faster & More Reliable Tuning of Neural Networks: Bayesian Optimization with Importance Sampling.
Hassim et al. Optimizing functional link neural network learning using modified bee colony on multi-class classifications
CN117439053A (en) Method, device and storage medium for predicting electric quantity of Stacking integrated model
CN107038244A (en) A kind of data digging method and device, a kind of computer-readable recording medium and storage control
CN116542701A (en) Carbon price prediction method and system based on CNN-LSTM combination model
Hu et al. A variable batch size strategy for large scale distributed dnn training

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20180615