CN111625321A - Virtual machine migration planning and scheduling method based on temperature prediction, system and medium thereof - Google Patents
Virtual machine migration planning and scheduling method based on temperature prediction, system and medium thereof Download PDFInfo
- Publication number
- CN111625321A CN111625321A CN202010748491.3A CN202010748491A CN111625321A CN 111625321 A CN111625321 A CN 111625321A CN 202010748491 A CN202010748491 A CN 202010748491A CN 111625321 A CN111625321 A CN 111625321A
- Authority
- CN
- China
- Prior art keywords
- cloud server
- virtual machine
- temperature
- module
- migration
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
- G06F18/24155—Bayesian classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2119/00—Details relating to the type or aim of the analysis or the optimisation
- G06F2119/08—Thermal analysis or thermal optimisation
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Computer Hardware Design (AREA)
- Geometry (AREA)
- Probability & Statistics with Applications (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention provides a virtual machine migration planning scheduling method based on temperature prediction, a system and a medium thereof, wherein a cloud server temperature initial prediction model is obtained by training historical data of a cloud server, newly added training data is obtained in real time in the scheduling process, the temperature prediction model is updated in real time, and the temperature of the cloud server is predicted in real time; determining a cloud server needing to be migrated out of the virtual machine according to the current real-time predicted cloud server temperature and cloud server load; comprehensively considering the client attribute of the virtual machine and the workload cost of virtual machine migration to determine the virtual machine to be migrated; the virtual machine is migrated to the running state cloud server with the lowest predicted temperature and meeting the load requirement, and the temperature of the running state cloud server and the number of the running state cloud servers are minimized, so that the energy-saving effect of the cloud data center is achieved.
Description
Technical Field
The invention relates to the field of infrastructure as a service (IaaS) cloud data center energy conservation, in particular to a virtual machine migration planning and scheduling method based on temperature prediction, a system and a medium thereof. Especially, the energy-saving technical scheme for minimizing the temperature and the number of the cloud servers in the running state is considered at the same time.
Background
With the continuous development of the information industry and the social economy, the scale of the IaaS cloud computing as a new resource use and delivery service mode tends to increase rapidly. As a bearer of the IaaS cloud computing service, energy consumption of the IaaS cloud data center has become a bottleneck affecting the cost of the IaaS cloud computing service.
In a traditional distributed computing system, the computing amount and/or the computing duration of computing tasks are often known in advance, and the computing system can plan the distribution of the computing tasks in a server in advance according to the computing amounts of a plurality of computing tasks so as to achieve the energy-saving goal of lowest total energy consumption of the server; however, for the IaaS cloud data center, the computing task of the user is unknown, and therefore, the migration path of the virtual machine can be planned in real time only by monitoring the actual use condition of the IaaS cloud data center cloud server to achieve energy saving of the IaaS cloud data center.
However, in the virtual machine migration scheduling scheme in the prior art, scheduling planning is usually implemented according to load information of a current cloud server, rather than considering a thermal load (i.e., a temperature) of the cloud server, and a virtual machine migration scheduling policy cannot be directly hooked with a data center energy-saving target. Or, in the prior art, when virtual machine migration scheduling is implemented, planning is implemented only by relying on sensing of the current temperature of the cloud server instead of predicting the future temperature of the cloud server, so that scheduling is delayed due to load change, and the cloud server load cannot be transferred effectively in time.
Therefore, a brand-new virtual machine migration scheduling planning method needs to be designed, the future temperature of the cloud server is used as a basis, the virtual machine is migrated to the cloud server with the low predicted temperature, and the temperature and the number of the cloud servers in the working state are minimized, so that the total energy consumption of the cloud servers of the IaaS cloud data center is reduced, and the purpose of saving energy of the IaaS cloud data center is finally achieved.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to provide a virtual machine migration planning and scheduling method based on temperature prediction, a system and a medium thereof.
The invention provides a virtual machine migration planning and scheduling method based on temperature prediction, which comprises the following steps:
step S1: the method comprises the steps of obtaining a cloud server list of a cloud data center, obtaining a predicted value of a Bayesian linear regression-based temperature prediction model for the temperature of each cloud server in the cloud server list of the cloud data center, and obtaining the predicted temperature of the cloud server;
step S2: defining the operating state cloud server with the load monitoring index meeting the low-load condition in the operating state cloud server list as a low-load cloud server, and constructing a low-load cloud server list;
step S3: traversing the low-load cloud server list, acquiring the virtual machines in each low-load cloud server, and listing the virtual machines into a to-be-migrated virtual machine list;
step S4: determining a target host cloud server of each virtual machine in the virtual machine list to be migrated according to the predicted temperature and the residual load capacity of each host cloud server in the host cloud server list, and completing virtual machine migration; the host cloud server comprises a normal operation state cloud server and cloud servers in a dormant state cloud server list; the selection sequence of the target host cloud server is that the cloud server in the normal operation state is preferentially selected, and when the cloud server in the normal operation state does not meet the matching condition, the cloud server in the dormant state is selected; the cloud server in the normal operation state is defined as a cloud server which is not listed in the low-load cloud server list and is not listed in the alarm cloud server list in the operation state cloud server list;
step S5: setting all the migrated low-load cloud servers of the virtual machines into a dormant state, listing an operating state cloud server list, and adding the operating state cloud server list into the dormant state cloud server list;
step S6: defining the operating state cloud server with the monitoring index meeting the alarm condition in the operating state cloud server list as an alarm cloud server, and listing all alarm cloud servers in an alarm cloud server list;
step S7: traversing the alarm cloud server list to obtain the virtual machine with the highest migration priority in each alarm cloud server, and listing the virtual machines into a to-be-migrated virtual machine list;
step S8: determining a target host cloud server of each virtual machine in the virtual machine list to be migrated according to the predicted temperature and the residual load capacity of each host cloud server in the host cloud server list, and completing virtual machine migration;
step S9: setting alarm cloud servers, from which the virtual machines are all migrated, to be in a dormant state, listing an operating state cloud server list, and adding the operating state cloud server list into the dormant state cloud server list;
step S10: judging whether the process is finished or not; if yes, ending the process; if not, the process goes to the step S1 to continue.
Preferably, the temperature prediction model in step S1 is obtained by:
step S101: establishing an initial training data set by using historical sampling data, and performing initial training on a Bayesian linear regression model to obtain an initial temperature prediction model;
step S102: in the daily operation process of the cloud data center, acquiring a newly added training data set in real time, performing online incremental training on a current cloud server temperature prediction model in an incremental mode, updating the temperature prediction model, and predicting the future temperature of the cloud server by using the temperature prediction model to obtain a predicted value of the future temperature of the cloud server;
wherein:
the step S101 includes the steps of:
step S10101: defining the time interval between every two adjacent sampling moments as a sampling period, defining a prediction period on each sampling period, wherein the length of the prediction period is the prediction depth of the temperature prediction model, and the initial time of the prediction period is the initial time of the same sampling period; the method comprises the steps that index data of a cloud server at the initial moment of a prediction period are used as model input data indexes, and the actually measured cloud server temperature at the end moment of the same prediction period is used as model output data indexes to construct an initial training data set; the measured cloud server temperature is defined as an arithmetic mean value of temperatures of different parts and/or components of the measured cloud server;
step S10102: preprocessing an initial training data set;
step S10103: training the Bayes linear regression model by utilizing the preprocessed initial training data set to obtain a parameter posterior distribution function of the initial Bayes linear regression model;
step S10104: outputting a parameter posterior distribution function of the initial Bayes linear regression model;
the step S102 includes the steps of:
step S10201: inputting a parameter posterior distribution function of a current Bayes linear regression model;
step S10202: acquiring a newly added training data set; wherein the newly added training data set comprises: index data of the cloud server at the initial time of a prediction period ending at the initial time of the current sampling period is used as an input data index, and the actually measured cloud server temperature of the cloud server at the end time of the prediction period ending at the initial time of the current sampling period is used as an output data index; the measured cloud server temperature is defined as an arithmetic mean value of temperatures of different parts and/or components of the measured cloud server;
step S10203: preprocessing the newly added training data set;
step S10204: performing incremental training on the current Bayes linear regression model by using the newly-added training data set after preprocessing, and updating a parameter posterior distribution function of the current Bayes linear regression model;
step S10205: obtaining a Bayesian optimal estimation of a parameter of a current Bayesian linear regression model;
step S10206: predicting the temperature of the cloud server at the end moment of the current prediction period based on the parameter Bayes optimal estimation of the current Bayes linear regression model;
step S10207: outputting the predicted temperature of the cloud server;
step S10208: judging whether the predicted temperature process of the cloud server is finished or not; if yes, ending the process; if not, the process goes to step S10202 to continue the process.
Preferably, in step S10101 and step S10202, the cloud server temperature prediction model training data index includes an input data index and an output data index;
the input data indicator includes any one or more of the following:
the number of CPU cores of the cloud server;
the cloud server uses the CPU core number;
the utilization rate of a CPU of the cloud server;
memory capacity of a cloud server;
the memory utilization rate of the cloud server;
current frequency of a cloud server CPU;
the cloud server inputs network flow;
the cloud server outputs network traffic;
the rotating speed of each fan of the cloud server;
the number of the virtual machines deployed by the cloud server nodes is increased;
the output data index comprises any one of the following data or an arithmetic mean of any plurality of data:
actually measuring the inlet air temperature of the cloud server;
actually measuring the air outlet temperature of the cloud server;
actually measuring the temperature of a CPU of the cloud server;
and (4) actually measuring the GPU temperature of the cloud server.
Preferably, the preprocessing in step S10102 and step S10203 includes the steps of:
a data cleaning step: if at the sampling instantObtained values of sampled input data itemsFor the vacancy value, filling the vacancy value with the historical average of the data item;
a normalization step: for each input data item, the minimum value and the maximum value of the data item in the initial training data set defined in the step S10102 are recorded as the minimum value and the maximum value respectively、Inputting the original value of the data itemMapped to approximate interval [0,1 ]]Is new value ofThe formula is as follows:
the value of the data item obtained after preprocessing each input data item in the training sampleAnd merging the preprocessed training sample input data as the input of the initial Bayesian linear regression training in the step S10103 and the incremental Bayesian linear regression training in the step S10204.
Preferably, the step S10103 includes: in BayesIn the initial training stage of linear regression, selecting model parameter prior distribution and giving initial hyper-parameter as,,After initial training of the Bayesian linear regression model is completed, the posterior distribution of model parameters obeys hyper-parameters of,,The gaussian-inverse gama probability distribution of (a), wherein n represents the last sampling period in the initial training dataset; hyper-parameter,,Are all made of,,And simple functions of the initial training sample set, with analytical solutions.
The step S10204 includes: in the Bayes linear regression increment training stage, the model parameter before the current sampling period t obeys hyper-parameter of,,In the Gaussian-inverse GAMMA probability distribution, the model parameter of the current sampling period t is updated into the obedience hyper-parameter by adding the sampling training sample of the sampling period t in the current sampling period t,,Gaussian-inverse gama probability distribution; hyper-parameter,,Are all made of,,And a simple function of the incremental training samples, with an analytical solution.
Preferably, the low load condition means: the CPU utilization rate of the cloud server in the running state is lower than a set first threshold value;
the alarm condition is satisfied, that is, any one of the following conditions is satisfied:
under a first condition, the utilization rate of a CPU (central processing unit) of the cloud server is higher than a set second threshold value, wherein the second threshold value is larger than the first threshold value;
in a second condition, the current predicted temperature of the cloud server is higher than the server alarm temperature.
Preferably, in the step S8, the virtual machine migration priority V is calculated according to the following steps:
step S801: obtaining a user level according to the monthly consumption amount or the annual consumption amount of the user; obtaining a user priority weighting coefficient R according to the user level; the user priority weighting factorIncluding values between 0 and 1;
step S802: obtaining the memory size of a virtual machine(ii) a Calculating the average value of the memory sizes of all virtual machines in the cloud server in the running state(ii) a According to the memory size of the virtual machineAnd the average value of the memory sizes of all virtual machines in all running state cloud serversCalculating the constant migration load weight coefficient of the virtual machine,;
Step S803: obtaining the memory change rate of the virtual machine in unit time(ii) a Calculating the average value of the memory change rates of all virtual machines in all running state cloud servers in unit time(ii) a According to the unit time memory change rate of the virtual machineAnd average value of memory change rate of all virtual machines in unit time in running state cloud serverCalculating the load weight coefficient of incremental migration of the virtual machine,;
Step S804: calculating the migration priority of the virtual machine according to the user priority weight coefficient, the virtual machine constant migration load weight coefficient and the virtual machine incremental migration load weight coefficient:
according to a migration priority calculation formula:wherein, in the step (A),,then virtual machine migration priority
Wherein the content of the first and second substances,represents a user priority weight coefficient and,representing a virtual machine constant migration load weight coefficient,representing a virtual machine incremental migration load weight coefficient,which represents the size of the memory of the virtual machine,represents the average memory size of all the virtual machines in all the running state cloud servers,represents the memory change rate of the virtual machine per unit time,represents the average memory change rate per unit time of all the virtual machines in all the running state cloud servers,the value range (0, 1) is an empirical parameter]。
Preferably, in step S4 and step S8, the step of determining the target host cloud server of each virtual machine in the to-be-migrated virtual machine list according to the predicted temperature and the remaining load capacity of each host cloud server in the host cloud server list all uses a minimum temperature server allocation principle algorithm, which specifically includes the following steps:
step A: traversing the virtual machines to be migrated in the virtual machine list to be migrated, and executing a first loop step for each virtual machine to be migrated, wherein the first loop step comprises the following steps:
step a: traversing all the cloud servers in the normal operation state, and acquiring the current preset temperature of each cloud server in the normal operation state;
step b: sorting the normally-operated cloud servers from low to high according to the predicted temperature to obtain a sorted list of cloud servers to be selected;
step c: traversing the sorted cloud server list to be selected, and executing a second circulation step, wherein the second circulation step comprises the following steps:
step c 1: for the current cloud server to be selected, judging whether the current cloud server to be selected meets the following conditions:
the predicted temperature is less than the server alarm temperature threshold;
the number of the residual CPU cores is larger than the CPU core number requirement of the virtual machine to be migrated;
the amount of the residual unallocated memory is greater than the memory requirement of the virtual machine to be migrated;
if so, distributing the virtual machine to be migrated to the current cloud server to be selected to obtain a target host cloud server; exiting the second loop step;
if not, go to step c2 to continue execution;
step c 2: judging whether a cloud server list to be selected has a next cloud server to be selected; if yes, go to step c3 to continue execution; if not, jumping to the step c4 to continue execution;
step c 3: taking the next cloud server to be selected as the current cloud server to be selected, and jumping to the step c1 to continue execution;
step c 4: listing a cloud server from the dormant state cloud server list, listing the cloud server into the normal operation state cloud server list, taking the cloud server as the current cloud server to be selected, and jumping to the step c1 to continue execution;
and B: judging whether a next virtual machine to be migrated exists or not; if not, entering the step C to continue execution; if yes, entering the step A to continue executing the first circulation step;
and C: and obtaining the matching relation between the virtual machine to be migrated and the target host cloud server.
The invention provides a virtual machine migration planning and scheduling system based on temperature prediction, which comprises:
the temperature prediction model training prediction module 100: training a temperature prediction model based on Bayesian linear regression, and predicting the temperature by using the temperature prediction model to obtain the predicted temperature of the cloud server;
cloud server classification module 200: classifying the cloud servers into an operating state cloud server and a dormant state cloud server according to the working state, wherein the operating state cloud server is classified into a low-load cloud server, a normal operating state cloud server and an alarm state cloud server;
the virtual machine migration module 300: based on the classification of the cloud server by the cloud server classification module 200, according to the predicted temperature of the cloud server obtained by the temperature prediction model training prediction module 100, migrating the virtual machine on the low-load cloud server and the virtual machine with the highest migration priority on the alarm cloud server to the host cloud server; the host cloud server comprises a normal operation state cloud server and cloud servers in a dormant state cloud server list; the selection sequence of the target host cloud server is that the cloud server in the normal operation state is preferentially selected, and when the cloud server in the normal operation state does not meet the matching condition, the cloud server in the dormant state is selected; the cloud server in the normal operation state is defined as a cloud server which is not listed in the low-load cloud server list and is not listed in the alarm cloud server list in the operation state cloud server list.
Preferably, the temperature prediction model training prediction module 100 comprises:
the data acquisition module 110: acquiring index data of a cloud server and actually measured cloud server temperature data;
the data preprocessing module 120: preprocessing the data acquired by the data acquisition module 110;
the temperature prediction model training module 130: training a Bayesian linear regression model by using the data preprocessed by the data preprocessing module 120;
the temperature prediction module 140: and predicting the temperature according to the current Bayesian linear regression model obtained by training of the temperature prediction model training module 130 to obtain the predicted temperature of the cloud server.
Preferably, the data acquisition module 110 includes:
initial training data set construction module 111: constructing an initial training data set by using historical sampling data; defining the time interval between every two adjacent sampling moments as a sampling period, defining a prediction period on each sampling period, wherein the length of the prediction period is the prediction depth of the temperature prediction model, and the initial time of the prediction period is the initial time of the same sampling period; the method comprises the steps that index data of a cloud server at the initial moment of a prediction period are used as model input data indexes, and the actually measured cloud server temperature at the end moment of the same prediction period is used as model output data indexes to construct an initial training data set; the measured cloud server temperature is defined as an arithmetic mean value of temperatures of different parts and/or components of the measured cloud server;
the newly added training data set construction module 112: acquiring a newly added training data set; wherein the newly added training data set comprises: index data of the cloud server at the initial time of a prediction period ending at the initial time of the current sampling period is used as a model input data index, and the actually measured cloud server temperature of the cloud server at the end time of the prediction period ending at the initial time of the current sampling period is used as a model output data index; the measured cloud server temperature is defined as an arithmetic mean value of temperatures of different parts and/or components of the measured cloud server;
the cloud server temperature prediction model training data indexes comprise input data indexes and output data indexes;
the input data indicator includes any one or more of the following:
the number of CPU cores of the cloud server;
the cloud server uses the CPU core number;
the utilization rate of a CPU of the cloud server;
memory capacity of a cloud server;
the memory utilization rate of the cloud server;
current frequency of a cloud server CPU;
the cloud server inputs network flow;
the cloud server outputs network traffic;
the rotating speed of each fan of the cloud server;
the number of the virtual machines deployed by the cloud server is increased;
the output data index comprises any one of the following data or an arithmetic mean of any plurality of data:
actually measuring the inlet air temperature of the cloud server;
actually measuring the air outlet temperature of the cloud server;
actually measuring the temperature of a CPU of the cloud server;
and (4) actually measuring the GPU temperature of the cloud server.
Preferably, the data preprocessing module 120 includes:
initial training data set preprocessing module 121: preprocessing an initial training data set;
the newly added training data set preprocessing module 122: preprocessing the newly added training data set;
the initial training data set preprocessing module 121 and the newly added training data set preprocessing module 122 both include:
the data cleaning module 1201: if at the sampling instantObtained values of sampled data itemsFor the vacancy value, filling the vacancy value with the historical average of the data item;
the normalization module 1202: for each data item, respectively recording the minimum value and the maximum value of the data item in the initial training data set as、The original value of the data itemMapped to approximate interval [0,1 ]]Is new value ofThe formula is as follows:
the value of the data item obtained after preprocessing each data item in the training sampleThe preprocessed training samples are combined as the input of the initial training and incremental training of the bayesian linear regression in the temperature prediction model training module 130.
Preferably, the temperature prediction model training module 130 includes:
initial training data set training module 131: performing initial training on the Bayesian linear regression model by using the preprocessed initial training data to obtain an initial temperature prediction model; wherein, in the initial training stage of Bayesian linear regression, the prior distribution of model parameters is selected to obey the given initial hyperparameter as,,After initial training of the Bayesian linear regression model is completed, the posterior distribution of model parameters obeys hyper-parameters of,,The gaussian-inverse gama probability distribution of (a), wherein n represents the last sampling period in the initial training dataset; hyper-parameter,,Are all made of,,And a simple function of the initial training sample set, with an analytic solution;
newly added training data set training module 132: performing online incremental training on the Bayesian linear regression model by using the newly-added training data set after preprocessing in an incremental manner; in the Bayesian linear regression increment training stage, the model parameter before the current sampling period t obeys hyper-parameter of,,Gauss-inverse ofGAMMA probability distribution, in the current sampling period t, updating the model parameter of the current sampling period t into the obedience hyper-parameter by adding the sampling training sample of the sampling period t,,Gaussian-inverse gama probability distribution; hyper-parameter,,Are all made of,,And a simple function of the incremental training samples, with an analytical solution.
Preferably, the cloud server classification module 200 includes:
the dormant state cloud server classifying module 201: classifying the cloud server in the dormant state as a dormant state cloud server; the dormant cloud server can be used as a host cloud server;
operating state cloud server classification module 202: classifying the cloud server in the running state into a cloud server in the running state;
the operating state cloud server classification module 202 includes:
the low-load cloud server classification module 20201: classifying cloud servers with the operating state cloud server node CPU utilization rate lower than a set first threshold value into low-load cloud servers;
the alarm cloud server classification module 20202: classifying the cloud server meeting the alarm condition as an alarm cloud server, wherein the meeting of the alarm condition means that any one of the following conditions is met:
under a first condition, the utilization rate of a CPU (central processing unit) of a cloud server node is higher than a set second threshold value, wherein the second threshold value is larger than the first threshold value;
a second condition, wherein the current predicted temperature of the cloud server is higher than the server alarm temperature;
normal operating state cloud server classification module 20203: and classifying the cloud servers which do not belong to the low-load cloud server and the alarm cloud server in the operation state cloud server list into normal operation state cloud servers, wherein the normal operation state cloud servers can be used as host cloud servers.
Preferably, the virtual machine migration module 300 includes:
cloud server temperature acquisition module 301: traversing the cloud server list of the cloud data center, and acquiring a cloud server temperature predicted value at the end moment of a prediction period defined on a sampling period of each cloud server at the current moment;
the to-be-migrated virtual machine obtaining module 302: traversing the low-load cloud servers, obtaining the virtual machines in each low-load cloud server as virtual machines to be migrated, and listing the virtual machines to be migrated in a virtual machine list to be migrated; traversing the alarm cloud servers to obtain a virtual machine with the highest migration priority in each alarm cloud server as a virtual machine to be migrated, and listing the virtual machine to be migrated in a virtual machine list to be migrated;
the migration operation module 303: determining a target host cloud server of each virtual machine in the virtual machine list to be migrated according to the predicted temperature and the residual load capacity of each host cloud server, and completing virtual machine migration;
the flow end determination module 304: judging whether the process is finished or not; if yes, ending the process; if not, triggering the cloud server temperature obtaining module 301 to enter the next virtual machine migration planning period, and continuing to obtain the cloud server predicted temperature.
Preferably, in the to-be-migrated virtual machine obtaining module 302, the virtual machine migration priority V is calculated through the following modules:
the user priority weighting coefficient obtaining module 30201: obtaining a user level according to the monthly consumption amount or the annual consumption amount of the user; obtaining a user priority weighting coefficient R according to the user level; the user priority weighting coefficient R comprises a numerical value between 0 and 1;
the virtual machine constant migration load weight coefficient obtaining module 30202: obtaining the memory size of a virtual machineP(ii) a Calculating the average value of the memory sizes of all virtual machines in the cloud server in the running state(ii) a According to the memory size of the virtual machinePAnd the average value of the memory sizes of all virtual machines in all running state cloud serversCalculating the constant migration load weight coefficient of the virtual machine,;
The virtual machine incremental migration load weight coefficient obtaining module 30203: obtaining the memory change rate of the virtual machine in unit timeS(ii) a Calculating the average value of the memory change rates of all virtual machines in all running state cloud servers in unit time(ii) a According to the unit time memory change rate of the virtual machineSAnd average value of memory change rate of all virtual machines in unit time in running state cloud serverCalculating the load weight coefficient of incremental migration of the virtual machine,;
The migration priority acquisition module 30204 of the virtual machine: calculating the migration priority of the virtual machine according to the user priority weight coefficient, the virtual machine constant migration load weight coefficient and the virtual machine incremental migration load weight coefficient:
according to a migration priority calculation formula:wherein, in the step (A),,then virtual machine migration priority
Wherein R represents a user priority weighting factor,representing a virtual machine constant migration load weight coefficient,represents the load weight coefficient of the incremental migration of the virtual machine, P represents the memory size of the virtual machine,represents the average memory size of all the virtual machines in all the running state cloud servers,represents the memory change rate of the virtual machine per unit time,represents the average memory change rate per unit time of all the virtual machines in all the running state cloud servers,the value range (0, 1) is an empirical parameter]。
Preferably, the migration operation module 303 includes:
the migration matching relationship assignment module 3031: traversing the virtual machines to be migrated in the virtual machine list to be migrated, and distributing target host cloud servers to the virtual machines to be migrated;
the migration matching relationship output module 3032: migrating the virtual machine to be migrated according to the matching relationship between the virtual machine to be migrated and the target host cloud server, which is established by the migration matching relationship allocation module 3031;
wherein the migration matching relationship assignment module 3031 includes:
predicted temperature ordering module 303101: sorting the normally-running cloud servers from low to high according to the predicted temperature obtained by the cloud server temperature obtaining module 301 to obtain a sorted list of cloud servers to be selected;
virtual machine to be migrated traverse module 303102: traversing the virtual machine to be migrated in the virtual machine list to be migrated, and acquiring the next virtual machine in the virtual machine list to be migrated as the current virtual machine to be migrated;
the to-be-selected cloud server screening module 303103: according to the order of the list of the cloud servers to be selected after the predicted temperature sorting module 303101 sorts, the cloud servers to be selected which meet the following conditions are allocated to the current virtual machine to be migrated as target host cloud servers:
the predicted temperature is less than the server alarm temperature threshold;
the number of the residual CPU cores is larger than the CPU core number requirement of the virtual machine to be migrated;
the amount of the residual unallocated memory is greater than the memory requirement of the virtual machine to be migrated;
sleep state cloud server wake-up module 303104: for the current to-be-migrated virtual machine to which the target host cloud server is not allocated by the to-be-selected cloud server screening module 303103, listing a cloud server from the dormant state cloud server list, listing the cloud server in the normal operation state cloud server list, and allocating the cloud server as the current to-be-selected cloud server to the to-be-migrated virtual machine.
According to the present invention, there is provided a computer readable storage medium storing a computer program, which when executed by a processor implements the steps of the virtual machine migration planning and scheduling method based on temperature prediction.
Compared with the prior art, the invention has the following beneficial effects:
1. the virtual machine is migrated to the running state cloud server which is low in predicted temperature and meets the requirements on the whole, and the temperature and the number of the running state cloud servers are minimized, so that the energy-saving effect of the cloud data center is achieved;
2. according to the cloud server migration method and the cloud server migration system, the cloud server of the virtual machine to be migrated is determined according to factors such as the current real-time predicted cloud server temperature and cloud server load, the cloud server failure risk is reduced, and the cloud server reliability is improved;
3. the cloud server temperature prediction module predicts based on a Bayesian linear regression model obtained by training, can realize online real-time training, and can predict the temperature of the cloud server in the short future in an online real-time manner;
4. the virtual machine to be migrated is determined by comprehensively considering the client attribute of the virtual machine and the migration cost of the virtual machine, so that the service interruption time of virtual machine migration is reduced, and the user experience is optimized.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:
FIG. 1 is a flowchart illustrating the overall steps of a method for scheduling virtual machine migration planning based on temperature prediction;
FIG. 2 is a flowchart illustrating the detailed steps of initial training of a Bayesian linear regression model;
FIG. 3 is a flowchart of the steps involved in incremental training and prediction of a Bayesian linear regression model;
FIG. 4 is a cloud server temperature prediction model training data list and a model training schematic;
FIG. 5 is a flowchart illustrating specific steps of training data preprocessing of a cloud server temperature prediction model;
FIG. 6 is a flowchart illustrating the steps involved in computing the migration priority of a virtual machine;
FIG. 7 is a flowchart illustrating specific steps of a minimum temperature server allocation rule algorithm;
FIG. 8 is a schematic diagram of the overall structure of a virtual machine migration planning and scheduling system based on temperature prediction;
FIG. 9 is a schematic diagram of a specific structure of a virtual machine migration planning and scheduling system based on temperature prediction;
FIG. 10 is a schematic diagram of a data preprocessing module 120 of the virtual machine migration planning and scheduling system based on temperature prediction;
fig. 11 is a schematic structural diagram of a module for determining a virtual machine migration priority by the virtual machine to be migrated obtaining module 302 of the virtual machine migration planning and scheduling system based on temperature prediction;
fig. 12 is a schematic structural diagram of the virtual machine migration planning scheduling system migration operation module 303 based on temperature prediction.
The figures show that:
a temperature prediction model training prediction module 100;
a data acquisition module 110;
an initial training data set construction module 111;
a newly added training data set construction module 112;
a data pre-processing module 120;
an initial training data set preprocessing module 121;
a newly added training data set preprocessing module 122;
a data cleaning module 1201;
a normalization module 1202;
a temperature prediction model training module 130;
an initial training data set training module 131;
a new training data set training module 132;
a temperature prediction module 140;
a cloud server classification module 200;
a dormant state cloud server classification module 201;
an operating state cloud server classification module 202;
a low-load cloud server classification module 20201;
the alarm cloud server classification module 20202;
the normal operation state cloud server classification module 20203;
a virtual machine migration module 300;
a cloud server temperature acquisition module 301;
a virtual machine to be migrated obtaining module 302;
a user priority weight coefficient acquisition module 30201;
a virtual machine constant migration load weight coefficient obtaining module 30202;
a virtual machine incremental migration load weight coefficient obtaining module 30203;
a migration priority acquisition module 30204 of the virtual machine;
a migration operation module 303;
a migration matching relationship assignment module 3031;
a predicted temperature sort module 303101;
the virtual machine to be migrated traverses module 303102;
a to-be-selected cloud server screening module 303103;
a sleep state cloud server wake-up module 303104;
a migration matching relationship output module 3032;
flow ends decision block 304.
Detailed Description
The present invention will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the invention, but are not intended to limit the invention in any way. It should be noted that it would be obvious to those skilled in the art that various changes and modifications can be made without departing from the spirit of the invention. All falling within the scope of the present invention.
First embodiment
In a first embodiment, a virtual machine migration planning and scheduling method based on temperature prediction according to the present invention is described with reference to fig. 1 to 7.
As shown in fig. 1, the virtual machine migration planning and scheduling method based on temperature prediction includes the following steps.
Step S1: the method comprises the steps of obtaining a cloud server list of a cloud data center, obtaining a predicted value of a Bayesian linear regression-based temperature prediction model for the temperature of each cloud server in the cloud server list of the cloud data center, and obtaining a predicted temperature of the cloud server; then, the flow proceeds to step S2 to continue execution.
Specifically, the temperature prediction model is obtained by performing initial training and incremental training on a bayesian linear regression model, and the current bayesian linear regression model constitutes a current temperature prediction model. In the initial training process, the mapping relation between the cloud server data and the temperature prediction model can be independently constructed and trained for each cloud server, and the training data of the temperature prediction model are acquired from the cloud server; in order to improve the prediction accuracy of the temperature prediction model, historical data in the initial training data set are historical data of the same cloud server, and the historical data of the same cloud server is used for training the temperature prediction model of the cloud server, so that the prediction accuracy can be improved. In the initial training process, the mapping relation between the cloud server data and the temperature prediction model can also be used for constructing and training a temperature prediction model for a plurality of cloud servers, the training data of the temperature prediction model is acquired from the plurality of cloud servers, and the data of the plurality of cloud servers are regarded as the data of one cloud server in different prediction periods during training.
More specifically, the predicted temperature of the cloud server will be used in the subsequent steps S4 and S8 as an important consideration for the virtual machine migration scheduling plan. The predicted temperature of the cloud server comprises a predicted temperature of the cloud server in an operating state and a predicted temperature of the cloud server in a dormant state.
Step S2: and defining the operation state cloud server with the load monitoring index meeting the low-load condition in the operation state cloud server list as a low-load cloud server, and constructing a low-load cloud server list. Then, the flow proceeds to step S3 to continue execution. The low-load condition is that: and the CPU utilization rate of the cloud server in the running state is lower than a set first threshold value. In the present application example, the first threshold value is set to 6%.
Specifically, after all the virtual machines on the low-load cloud server are migrated, the virtual machines can be adjusted to a dormant state to become a dormant cloud server so as to save energy. Therefore, a low-load cloud server list is constructed in step S2, then migration of part or all of the virtual machines in the low-load cloud server list is completed in step S3 and step S4, and the low-load cloud servers from which all the virtual machines have been migrated are adjusted to a sleep state. As can be understood by those skilled in the art, the running state cloud server list is used to indicate the cloud servers currently in a running state, and the cloud servers not currently in a running state are excluded from being moved out of the running state cloud server list; the low-load cloud server list is used for indicating the cloud servers which are currently in the low-load operation state, and the cloud servers which are not currently in the low-load operation state are excluded from being moved out of the low-load cloud server list.
Step S3: and traversing the low-load cloud server list, acquiring the virtual machines in each low-load cloud server, and listing the virtual machines into a to-be-migrated virtual machine list.
Specifically, as will be understood by those skilled in the art, the to-be-migrated virtual machine list is used to indicate the virtual machines currently in the to-be-migrated state, and the virtual machines not currently in the to-be-migrated state are excluded from being moved out of the to-be-migrated virtual machine list.
Step S4: determining a target host cloud server of each virtual machine in the virtual machine list to be migrated according to the predicted temperature and the residual load capacity of each host cloud server in the host cloud server list, and completing virtual machine migration; the flow advances to step S5 to continue execution.
Specifically, the host cloud server is an operating state cloud server and does not belong to a low-load cloud server, or the host cloud server may also be a dormant state cloud server. The selection sequence of the host cloud server is that the cloud server in the normal operation state is preferentially selected, and when the cloud server in the normal operation state does not meet the matching condition, the cloud server in the dormant state is selected; the matching condition may be, for example, that an index of the remaining load capacity of the cloud server is greater than a threshold value of a demand of the virtual machine to be migrated for the index of the available load capacity of the host server. As will be appreciated by those skilled in the art, the host cloud server list is used to indicate the current host cloud server, and cloud servers that do not currently conform to the host cloud server definition will be excluded from being moved out of the host cloud server list.
More specifically, after the virtual machine is migrated from the low-load cloud server, for the virtual machine, a cloud server in a normal operation state with a low future predicted temperature is preferentially selected as a host cloud server in consideration of the migration situation of the virtual machine into the cloud server. However, if the current load capacity of the cloud server is full, the cloud server cannot be used as a host cloud server. And when the cloud server in the abnormal operation state meets the load capacity requirement of the virtual machine, selecting the cloud server in the dormant state as a host cloud server.
Step S5: and setting the low-load cloud servers, from which the virtual machines are all migrated, to be in a dormant state, listing a running state cloud server list, and adding the running state cloud server list into the dormant state cloud server list. Then, the flow proceeds to step S6 to continue execution.
Specifically, as will be understood by those skilled in the art, the dormant state cloud server list is used to indicate cloud servers currently in a dormant state, and cloud servers not currently in a dormant state are excluded from being moved out of the dormant state cloud server list.
Step S6: and defining the operating state cloud server with the monitoring index meeting the alarm condition in the operating state cloud server list as an alarm cloud server, and listing all alarm cloud servers in an alarm cloud server list. Then, the flow proceeds to step S7 to continue execution.
Specifically, in step S6, the alarm condition being satisfied means that any one of the following conditions is satisfied: under a first condition, the utilization rate of a CPU (central processing unit) of the cloud server is higher than a set second threshold value, wherein the second threshold value is larger than the first threshold value; in the present application example, the first threshold is set to 6%, and the second threshold is set to 90%; in the second condition, the current predicted temperature of the cloud server is higher than the server alarm temperature, which is set to 95 degrees in this application example.
Further, the alarm cloud server is usually high in load, and is prone to failure caused by continuous high temperature and other factors after long-term operation, and load reduction needs to be considered. Therefore, the alarm cloud server list is constructed in step S6, then the migration of the virtual machine with the highest migration priority in the alarm cloud server list is completed in step S7 and step S8, and the cloud server that does not meet the definition of the alarm cloud server after the virtual machine is migrated is excluded from being migrated out of the alarm cloud server list.
Step S7: and traversing the alarm cloud server list to obtain the virtual machine with the highest migration priority in each alarm cloud server, and listing the virtual machine with the highest migration priority in the alarm cloud server into a to-be-migrated virtual machine list. Then, the flow proceeds to step S8 to continue execution.
Specifically, as will be understood by those skilled in the art, the list of virtual machines to be migrated is used to indicate the virtual machines to be migrated, and the migrated virtual machines will be excluded from being moved out of the list of virtual machines to be migrated.
Step S8: and determining a target host cloud server of each virtual machine in the virtual machine list to be migrated according to the predicted temperature and the residual load capacity of each host cloud server in the host cloud server list, and completing virtual machine migration. Then, the flow proceeds to step S9 to continue execution.
More specifically, referring to the description of step S4, before the virtual machine migrates into a cloud server, the cloud server with the low future predicted temperature is preferentially selected as the host cloud server in consideration of the migration situation of the virtual machine into the cloud server. However, if the current load capacity of the cloud server is full, the cloud server cannot be used as a host cloud server. And when the cloud server in the abnormal operation state meets the load capacity requirement of the virtual machine, selecting the cloud server in the dormant state as a host cloud server.
Step S9: and setting the alarm cloud servers, from which the virtual machines are all migrated, to be in a dormant state, listing an operating state cloud server list, and adding the operating state cloud server list into the dormant state cloud server list. Then, the flow proceeds to step S10 to continue execution.
Specifically, as will be understood by those skilled in the art, the dormant state cloud server list is used to indicate cloud servers currently in a dormant state, and cloud servers not currently in a dormant state are excluded from being moved out of the dormant state cloud server list.
Step S10: judging whether the process is finished or not; if yes, ending the process; if not, jumping to the step S1 to continue executing, and implementing a new round of virtual machine scheduling planning.
The monitoring period of the working state of the cloud server is also a virtual machine migration planning period of the virtual machine, and one virtual machine migration planning period of the virtual machine is defined as one complete execution of the steps 1 to 9 of the virtual machine migration planning and scheduling method based on temperature prediction. The time consumed by the monitoring period of the working state of one cloud server in the application example is usually between 2 and 5 minutes. Setting the length of the prediction period of the temperature prediction model according to the empirical value can enable the monitoring period of the working state of the cloud server to be shorter than the prediction period of the temperature prediction model, for example, the experience time of one monitoring period is shorter than the prediction period of 6 minutes between 2 and 5 minutes, so that the migration progress of the virtual machine is ensured to be matched with the prediction depth of the future thermal load of the host cloud server. In the application example, the sampling period of the temperature prediction model is set to be shorter than the shortest experience time of the monitoring period of the working state of the cloud server, for example, the experience time of the monitoring period is between 2 and 5 minutes and is longer than the sampling period of 30 seconds set by the application example, so that the real-time and effective predicted value of the transient future temperature of the cloud server can be always obtained by monitoring the working state of the cloud server. As can be understood by those skilled in the art, the initial training of the temperature prediction model belongs to the preparation work before the scheduling of the virtual machine migration plan, and only one time of initial training is needed.
Further, the temperature prediction model in step S1 is obtained by the following steps.
Step S101: establishing an initial training data set by using historical sampling data, and performing initial training on a Bayesian linear regression model to obtain an initial temperature prediction model; then, the process proceeds to step S102.
Step S102: in the daily operation process of the cloud data center, a newly added training data set is collected in real time, online incremental training is carried out on a current cloud server temperature prediction model in an incremental mode, the temperature prediction model is updated, the future temperature of the cloud server is predicted by using the temperature prediction model, and a predicted value of the future temperature of the cloud server is obtained. The temperature prediction model is used for predicting the transient future temperature of the cloud server, for example, predicting the temperature at the future 6 minutes.
In a preferred embodiment, in the prediction of the future temperature of the cloud server, statistical reasoning is performed by using a Bayesian linear regression model, Gaussian-inverse GAMMA probability distribution is selected as the Bayesian linear regression model parameter conjugate prior distribution, and the posterior distribution of the Bayesian linear regression model parameters obtained by carrying out Bayesian reasoning by using initial training data is also Gaussian-inverse GAMMA probability distribution. The prior distribution and the posterior distribution of the parameters of the Bayesian linear regression model are the same distribution, so that the online rapid incremental training of the Bayesian linear regression model in the step S102 becomes possible, and the purpose of rapidly, efficiently and accurately predicting the temperature of the cloud server is achieved.
The bayesian linear regression model is explained in detail below.
I.e. the set of input samples of the Bayesian linear regression modelConsisting of n successive sample period input samples, here represented by symbols:。In the form of a matrix representation.The input sample obtained in the nth sampling period is shown, and in the application example of the present invention, the input sample data includes index data of a cloud server. Bayesian linear regression model output sample setConsists of n consecutive sample period output samples, here denoted by:。the output sample obtained in the nth sampling period is represented, and in the application example of the present invention, the output sample is actually measured cloud server temperature data, specifically, an arithmetic average of actually measured cloud server temperature at different parts and/or components.
The Bayesian linear regression model is as follows:
representing a set of samples at a given inputBayesian linear regression model parametersAndnext, a sample set is outputIs determined as a conditional probability density function. Outputting the sample set as described aboveObeys an n-dimensional gaussian distribution.
WhereinA weight vector that is a feature of the input sample;is the variance of the linear regression residual;,i.e., the parameters that make up the bayesian linear regression model.
Next, the parameters of the Bayesian linear regression model are alignedAndthe prior probability distribution of (a) is illustrated.
Conjugate prior probability based on Bayesian inference principleCan be decomposed intoSelecting the hyper-parameter as,,Gauss-inverse GAMMA probability distribution as parameterAndprior probability distribution. Wherein the varianceIs of the following formula:
variance (variance)Obey inverse GAMMADistributing; parameter(s)Andis a hyper-parameter of inverse GAMMA distribution; the value of the over-parameter is selected as,。
given variance,Obey gaussDistributing; expectation vectorVariance matrixA hyper-parameter of k-dimensional Gaussian distribution; selection of the value of the hyper-parameter in the application example of the invention:
is composed ofA dimension 0 vector; parameter(s)To representAn order identity matrix;representing a set of input samples as an input quantityThe characteristic dimension of (c).
The posterior distribution of the model parameters is calculated by:
wherein the content of the first and second substances,is given set of input samplesAnd outputting the sample setA posterior probability density function of the model parameters of time. Likelihood that the posterior probability density equals the outputMultiplying by a parameter of a given inputPrior probability density function ofAnd divided by a normalization constant.
Specifically to bayesian linear regression, the posterior distribution is:
=
In summary, the model observes the input sample setAnd outputting the sample setAfter, parameterAndthe posterior probability distribution obeys a hyper-parameter of,,The hyper-parameter calculation formula of the Gauss-inverse GAMMA distribution is as follows:
Through the Bayes statistical reasoning, Bayes linear regression inputs characteristic weight parametersThe initial training optimal Bayesian estimation is as follows:
expressing the predicted value of the cloud server temperature at the end moment of the prediction period m by using the initial Bayes optimal linear regression model asWhereinA predicted value indicating the cloud server temperature at the end of the prediction period m,input cloud server index data indicating the start time of the prediction period m.
The Bayesian linear regression incremental training will be explained in detail below.
In the case where the input sample number n >1, bayesian incremental training is represented by:
wherein the content of the first and second substances,for observing all historical input sample sets in sampling period tAnd all historical output sample setsModel parameters under the conditions ofIs determined. This term equals the input sample given the current sampling periodOutputting samples for the current sampling periodLikelihood ofMultiplying the set of all historical input samples observed in sampling period n-1And all historical output sample setsModel parameters under the conditions ofProbability density function ofAnd is normalized.
Particularly, Bayesian linear regression is adopted, and when the sampling period t-1 is set, the model parametersThe posterior probability density function is hyperparameter,,,The Gauss-inverse GAMMA distribution of the training sample is observed in the sampling period t,Model parametersThe posterior probability density function is hyperparameter,,,Gaussian-inverse GAMMA distribution. The hyper-parameter update formula, i.e. the incremental training formula, is as follows:
passing through the above shellfishThe Bayes linear regression is used for inputting the characteristic weight parameters in the current sampling period t through the Bayes statistical inferenceThe optimal Bayesian estimation of incremental training is as follows:
the prediction of the cloud server temperature at the end of the current prediction period t by using the current incremental bayesian optimal linear regression model is represented by formula 1:
WhereinA predicted value indicating the cloud server temperature at the end of the current prediction period t,cloud server index data representing model inputs at the start of the current prediction period t.
As shown in fig. 2, specifically, the step S101 includes the following steps.
Step S10101: defining the time interval between every two adjacent sampling moments as a sampling period, defining a prediction period on each sampling period, wherein the length of the prediction period is the prediction depth of the temperature prediction model, and the initial time of the prediction period is the initial time of the same sampling period; the method comprises the steps that index data of a cloud server at the initial moment of a prediction period are used as model input data indexes, and the actually measured cloud server temperature at the end moment of the same prediction period is used as model output data indexes to construct an initial training data set; the measured cloud server temperature is defined as an arithmetic mean value of temperatures of different parts and/or components of the measured cloud server; then, the process proceeds to step S10102.
Specifically, the initial time of two adjacent prediction periods is the initial time of two adjacent sampling periods. In a preferred embodiment, one prediction period is composed of a plurality of consecutive sampling periods, and the initial time of the prediction period is the same as the initial time of the first sampling period; the initial time of two consecutive adjacent prediction periods is the initial time of two consecutive adjacent sampling periods respectively. For example, assume that there are ten sampling instants, denoted T1, T2, …, T10; the interval time between each sampling time and the next sampling time forms a sampling period, the initial time of the sampling period is the sampling time, and the end time of the sampling period is the next sampling time. Further assuming that a prediction period consists of five sample periods, the first prediction period is the interval between T1 to T6, the second prediction period is the interval between T2 to T7, the third prediction period is the interval between T3 to T8, and so on.
Step S10102: preprocessing an initial training data set; then, the process proceeds to step S10103.
Step S10103: training the Bayes linear regression model by utilizing the preprocessed initial training data set to obtain a parameter posterior distribution function of the initial Bayes linear regression model; then, the process proceeds to step S10104.
Specifically, the step S10103 includes: in the initial training stage of Bayesian linear regression, the prior distribution of model parameters is selected to obey the given initial hyperparameter as,,After initial training of the Bayesian linear regression model is completed, the posterior distribution of model parameters obeys hyper-parameters of,,The gaussian-inverse gama probability distribution of (a), wherein n represents the last sampling period in the initial training dataset; hyper-parameter,,Are all made of,,And simple functions of the initial training sample set, with analytical solutions.
Step S10104: and outputting the parameter posterior distribution function of the initial Bayes linear regression model.
As shown in fig. 3, specifically, the step S102 includes the following steps.
Step S10201: inputting a parameter posterior distribution function of a current Bayes linear regression model; then proceed to step S10202 to continue execution.
Step S10202: acquiring a newly added training data set; wherein the newly added training data set comprises: index data of the cloud server at the initial time of a prediction period ending at the initial time of the current sampling period is used as an input data index, and the actually measured cloud server temperature of the cloud server at the end time of the prediction period ending at the initial time of the current sampling period is used as an output data index; the measured cloud server temperature is defined as an arithmetic mean value of temperatures of different parts and/or components of the measured cloud server; then proceed to step S10203 to continue execution.
Step S10203: preprocessing the newly added training data set; then proceed to step S10204 to continue execution.
Step S10204: performing incremental training on the current Bayes linear regression model by using the newly-added training data set after preprocessing, and updating a parameter posterior distribution function of the current Bayes linear regression model; then proceed to step S10205 to continue execution.
Specifically, the step S10204 includes: in the Bayes linear regression increment training stage, the model parameter before the current sampling period t obeys hyper-parameter of,,In the Gaussian-inverse GAMMA probability distribution, the model parameter of the current sampling period t is updated into the obedience hyper-parameter by adding the sampling training sample of the sampling period t in the current sampling period t,,Gaussian-inverse gama probability distribution; hyper-parameter,,Are all made of,,And a simple function of the incremental training samples, with an analytical solution.
Step S10205: obtaining a Bayesian optimal estimation of a parameter of a current Bayesian linear regression model; then proceed to step S10206 to continue execution.
Step S10206: predicting the temperature of the cloud server at the end moment of the current prediction period based on the parameter Bayes optimal estimation of the current Bayes linear regression model; then proceed to step S10207 to continue execution.
Specifically, in the step S10206, the temperature of the cloud server at the end time of the prediction period defined on the current sampling period is predicted by using formula 1 based on the current bayesian linear regression model.
Step S10207: outputting the predicted temperature of the cloud server; then, the process proceeds to step S10208.
Step S10208: judging whether the predicted temperature process of the cloud server is finished or not; if yes, ending the process; if not, the process goes to step S10202 to continue the process.
As shown in fig. 4, the training data indexes of the cloud server temperature prediction model in steps S10101 and S10202 include an input data index and an output data index.
The input data indicator includes any one or more of the following: the number of CPU cores of the cloud server; the cloud server uses the CPU core number; the utilization rate of a CPU of the cloud server; memory capacity of a cloud server; the memory utilization rate of the cloud server; current frequency of a cloud server CPU; the cloud server inputs network flow; the cloud server outputs network traffic; the rotating speed of each fan of the cloud server; the number of the virtual machines deployed by the cloud server nodes is increased;
the output data index comprises any one of the following data or an arithmetic mean of any plurality of data: actually measuring the inlet air temperature of the cloud server; actually measuring the air outlet temperature of the cloud server; actually measuring the temperature of a CPU of the cloud server; and (4) actually measuring the GPU temperature of the cloud server.
As shown in fig. 5, the data in the initial training data set is not necessarily ideal, and there is a problem such as data loss, so that the data in the initial training data set needs to be preprocessed, and the preprocessing in step S10102 and step S10203 includes the following steps.
A data cleaning step: if at the sampling instantObtained values of sampled input data itemsFor the vacancy value, filling the vacancy value with the historical average of the data item;
a normalization step: for each input data item, the minimum value and the maximum value of the data item in the initial training data set defined in the step S10102 are recorded as the minimum value and the maximum value respectively、Inputting the original value of the data itemMapped to approximate interval [0,1 ]]Is new value ofThe formula is as follows:
the value of the data item obtained after preprocessing each input data item in the training sampleAnd merging the preprocessed training sample input data as the input of the initial Bayesian linear regression training in the step S10103 and the incremental Bayesian linear regression training in the step S10204.
As shown in fig. 6, in the step S8, the virtual machine migration priority V is calculated according to the following steps. The calculation principle of the migration priority comprehensively considers the level of the user and the workload cost of virtual machine migration. Lower user levels generally place less demands on the continuity of the computing task and therefore migrate preferentially. The smaller the constant and incremental migration costs of the virtual machine, the more preferred the migration.
Step S801: obtaining a user level according to the monthly consumption amount or the annual consumption amount of the user; obtaining a user priority weight coefficient according to the user level(ii) a The user priority weighting coefficient R comprises a numerical value between 0 and 1;
specifically, the user priority weighting coefficient R is determined according to the following user type. The customer type is VVIP, and the user priority weighting factor R is set to 0.2. The customer type is VIP and the user priority weighting factor R is set to 0.5. The customer type is normal (monthly spending no less than 50% of the mean) and the user priority weighting factor R is set to 0.7. The customer type is low quality (monthly spending amount is less than 50% of the mean) and the user priority weighting factor R is set to 1.
Step S802: obtaining the memory size of the virtual machineP(ii) a Calculating the average value of the memory sizes of all virtual machines in the cloud server in the running state(ii) a According to the memory size of the virtual machinePAnd all virtual machine memories in all running state cloud serversAverage value of sizeCalculating the constant migration load weight coefficient of the virtual machine,;
Step S803: obtaining the memory change rate of the virtual machine in unit timeS(ii) a Calculating the average value of the memory change rates of all virtual machines in all running state cloud servers in unit time(ii) a According to the unit time memory change rate of the virtual machineSAnd average value of memory change rate of all virtual machines in unit time in running state cloud serverCalculating the load weight coefficient of the incremental migration of the virtual machine,;
Step S804: calculating the migration priority of the virtual machine according to the user priority weight coefficient, the virtual machine constant migration load weight coefficient and the virtual machine incremental migration load weight coefficient:
according to a migration priority calculation formula:wherein, in the step (A),,then virtual machine migration priority
Wherein R represents a user priority weighting factor,representing a virtual machine constant migration load weight coefficient,represents the load weight coefficient of the incremental migration of the virtual machine, P represents the memory size of the virtual machine,represents the average memory size of all the virtual machines in all the running state cloud servers,represents the memory change rate of the virtual machine per unit time,represents the average memory change rate per unit time of all the virtual machines in all the running state cloud servers,the value range (0, 1) is an empirical parameter]In the application example, the empirical constant value is 0.0001.
As shown in fig. 7, in step S4 and step S8, the step of determining the target host cloud server of each virtual machine in the to-be-migrated virtual machine list according to the predicted temperature and the residual load capacity of each host cloud server in the host cloud server list both adopts a minimum temperature server allocation principle algorithm, and specifically includes the following steps.
Step A: and traversing the virtual machines to be migrated in the virtual machine list to be migrated, and executing a first circulation step for each virtual machine to be migrated. Specifically, a first loop step is executed for the current virtual machine to be migrated, and then the step B is entered for continuous execution.
And B: judging whether a next virtual machine to be migrated exists or not; if not, entering the step C to continue execution; if yes, taking the next virtual machine to be migrated as the current virtual machine to be migrated, and jumping to the step A to continue to execute the first loop step.
And C: and obtaining the matching relation between the virtual machine to be migrated and the target host cloud server.
Wherein the first circulating step includes the following steps.
Step a: traversing all the cloud servers in the normal operation state, and acquiring the current preset temperature of each cloud server in the normal operation state; then step b is entered for continuous execution.
Step b: sorting the normally-operated cloud servers from low to high according to the predicted temperature to obtain a sorted list of cloud servers to be selected; then step c is entered for continuous execution.
Step c: and traversing the sorted cloud server list to be selected, and executing a second circulation step.
Wherein the second circulating step includes the following steps.
Step c 1: for the current cloud server to be selected, judging whether the current cloud server to be selected meets the following conditions: the predicted temperature is less than the server alarm temperature threshold; the number of the residual CPU cores is larger than the CPU core number requirement of the virtual machine to be migrated; and the amount of the residual unallocated memory is greater than the memory requirement of the virtual machine to be migrated. If so, distributing the virtual machine to be migrated to the current cloud server to be selected to obtain a target host cloud server; exiting the second loop step; if not, go to step c2 to continue execution.
Step c 2: judging whether a cloud server list to be selected has a next cloud server to be selected; if yes, go to step c3 to continue execution; if not, go to step c4 to continue execution.
Step c 3: and taking the next cloud server to be selected as the current cloud server to be selected, and jumping to the step c1 to continue execution.
Step c 4: and c, listing a cloud server from the dormant state cloud server list, listing the cloud server into the normal operation state cloud server list, taking the cloud server as the current cloud server to be selected, and jumping to the step c1 to continue execution.
Second embodiment
In a second embodiment, a virtual machine migration planning and scheduling system based on temperature prediction according to the present invention is illustrated by fig. 8 to 12. A person skilled in the art may understand the first embodiment as a preferred embodiment of the second embodiment, i.e. the system in the second embodiment may be implemented by performing the method steps in the first embodiment.
As shown in fig. 8, the virtual machine migration planning and scheduling system based on temperature prediction includes:
the temperature prediction model training prediction module 100: training a temperature prediction model based on Bayesian linear regression, and predicting the temperature by using the temperature prediction model to obtain the predicted temperature of the cloud server;
cloud server classification module 200: classifying the cloud servers into an operating state cloud server and a dormant state cloud server according to the working state, wherein the operating state cloud server is classified into a low-load cloud server, a normal operating state cloud server and an alarm state cloud server;
the virtual machine migration module 300: based on the classification of the cloud server by the cloud server classification module 200, the virtual machine in the low-load cloud server and the virtual machine with the highest migration priority on the alarm cloud server are migrated to the normally-operating cloud server or the dormant cloud server according to the predicted temperature of the cloud server obtained by the temperature prediction model training prediction module 100. The predicted temperature of the cloud server comprises a predicted temperature of the cloud server in an operating state and a predicted temperature of the cloud server in a dormant state. The selection sequence of the cloud servers of the migration target host of the virtual machine is that the cloud servers in the normal running state are preferentially selected, and when the cloud servers in the normal running state do not meet the matching condition, the cloud servers in the dormant state are selected;
the temperature prediction model training prediction module 100 comprises:
the data acquisition module 110: acquiring index data of a cloud server and actually measured cloud server temperature data;
the data preprocessing module 120: preprocessing the data acquired by the data acquisition module 110;
the temperature prediction model training module 130: training a Bayesian linear regression model by using the data preprocessed by the data preprocessing module 120;
the temperature prediction module 140: predicting the temperature according to the current Bayesian linear regression model obtained by training of the temperature prediction model training module 130 to obtain the predicted temperature of the cloud server;
specifically, in an initial stage training stage of predicting the cloud server temperature by implementing bayesian linear regression, first, the data acquisition module 110 acquires index data of a historical cloud server and an actually measured cloud server temperature; sending the collected historical data to the data preprocessing module 120 for data preprocessing; secondly, the data preprocessing module 120 preprocesses the data acquired by the data acquisition module 110; sending the preprocessed data to a temperature prediction model training module 130 for initial training of a Bayesian linear regression model; finally, the temperature prediction model training module 130 initially trains the bayesian linear regression model using the preprocessed data generated by the data preprocessing module 120 to obtain an initial bayesian linear regression model.
In the stage of implementing bayesian linear regression online increment training and prediction, in each bayesian linear regression online increment training and prediction implementation period, firstly, the data acquisition module 110 acquires index data of a cloud server in a prediction period ending at the initial moment of the current sampling period and actually measured cloud server temperature as an increment training sample; sending the acquired incremental training data to the data preprocessing module 120 for data preprocessing; secondly, the data preprocessing module 120 preprocesses the incremental training samples acquired by the data acquisition module 110; sending the preprocessed incremental training samples to a temperature prediction model training module 130 for incremental training of a Bayesian linear regression model; then, the temperature prediction model training module 130 performs incremental training on the bayesian linear regression model by using the preprocessed incremental samples generated by the data preprocessing module 120, updates the bayesian linear regression model, and triggers the temperature prediction module 140 to predict the temperature of the cloud server by using the updated bayesian linear regression model of the temperature prediction model training module 130 after the updating is completed, so as to obtain a predicted value of the temperature of the cloud server. At this point, a Bayesian linear regression online increment training and prediction implementation period is completed. When the initial moment of the next sampling period comes, the temperature prediction module 140 triggers the data acquisition module 110 to start acquisition work, and starts the next bayesian linear regression online incremental training and prediction implementation period.
The bayesian linear regression model is explained in detail below.
I.e. the set of input samples of the Bayesian linear regression modelConsists of n consecutive sample period input samples, here denoted by the notation:。in the form of a matrix representation.The input sample obtained in the nth sampling period is shown, and in the application example of the present invention, the input sample data includes index data of a cloud server. Bayesian linear regression model output sample setConsists of n consecutive sample period output samples, here denoted by:。the output sample obtained in the nth sampling period is represented, and in the application example of the present invention, the output sample is actually measured cloud server temperature data, specifically an arithmetic average of actually measured temperatures of different parts and/or components of the cloud server.
The Bayesian linear regression model is as follows:
representing a set of samples at a given inputBayesian linear regression model parametersAndnext, a sample set is outputIs determined as a conditional probability density function. Outputting the sample set as described aboveObeys an n-dimensional gaussian distribution.
WhereinA weight vector that is a feature of the input sample;is the variance of the linear regression residual;,i.e., the parameters that make up the bayesian linear regression model.
Next, the parameters of the Bayesian linear regression model are alignedAndthe prior probability distribution of (a) is illustrated.
Conjugate prior probability based on Bayesian inference principleCan be decomposed intoSelecting the hyper-parameter as,,Gauss-inverse GAMMA probability distribution as parameterAndprior probability distribution. Wherein the varianceIs of the following formula:
variance (variance)Obey inverse GAMMADistributing; parameter(s)Andis a hyper-parameter of inverse GAMMA distribution; the value of the over-parameter is selected as,。
given variance,Obey gaussDistributing; desired directionMeasurement ofVariance matrixA hyper-parameter of k-dimensional Gaussian distribution; selection of the value of the hyper-parameter in the application example of the invention:
is composed ofA dimension 0 vector; parameter(s)To representAn order identity matrix;representing a set of input samples as an input quantityThe characteristic dimension of (c).
The posterior distribution of the model parameters is calculated by:
wherein the content of the first and second substances,is given set of input samplesAnd outputting the sample setA posterior probability density function of the model parameters of time. Likelihood that the posterior probability density equals the outputMultiplying by a parameter of a given inputPrior probability density function ofAnd divided by a normalization constant.
Specifically to bayesian linear regression, the posterior distribution is:
=
In summary, the model observes the input sample setAnd outputting the sample setAfter, parameterAndthe posterior probability distribution obeys a hyper-parameter of,,The hyper-parameter calculation formula of the Gauss-inverse GAMMA distribution is as follows:
Through the Bayes statistical reasoning, Bayes linear regression inputs characteristic weight parametersThe initial training optimal Bayesian estimation is as follows:
expressing the predicted value of the cloud server temperature at the end moment of the prediction period m by using the initial Bayes optimal linear regression model asWhereinIndicating the end of the prediction period mThe predicted value of the temperature of the cloud server,input cloud server index data indicating the start time of the prediction period m.
More specifically, bayesian linear regression incremental training is described in detail below.
In the case where the input sample number n >1, bayesian incremental training is represented by:
wherein the content of the first and second substances,for observing all historical input sample sets in sampling period tAnd all historical output sample setsModel parameters under the conditions ofIs determined. This term equals the input sample given the current sampling periodOutputting samples for the current sampling periodLikelihood ofMultiplying the set of all historical input samples observed in sampling period n-1And all history of inputSet of output samplesModel parameters under the conditions ofProbability density function ofAnd is normalized.
Particularly, Bayesian linear regression is adopted, and when the sampling period t-1 is set, the model parametersThe posterior probability density function is hyperparameter,,,The Gauss-inverse GAMMA distribution of the training sample is observed in the sampling period t,Model parametersThe posterior probability density function is hyperparameter,,,Gaussian-inverse GAMMA distribution. The hyper-parameter update formula, i.e. the incremental training formula, is as follows:
through the Bayes statistical reasoning, the Bayes linear regression inputs the characteristic weight parameter in the current sampling period tThe optimal Bayesian estimation of incremental training is as follows:
the prediction of the cloud server temperature at the end of the current prediction period t by using the current incremental bayesian optimal linear regression model is represented by formula 1:
WhereinA predicted value indicating the cloud server temperature at the end of the current prediction period t,cloud server index data representing model inputs at the start of the current prediction period t. The temperature prediction module 140 predicts the temperature of the cloud server at the end time of the prediction period defined on the current sampling period by using formula 1 based on the current bayesian linear regression model.
Further, in the stage of implementing virtual machine migration, the virtual machine migration module 300 first determines which virtual machines to be migrated to construct a virtual machine list to be migrated, then determines the order of migrating the virtual machines according to the virtual machine migration priority, then determines a target host cloud server according to the cloud server predicted temperature obtained by the temperature prediction module 140 and the residual load capacity of the cloud server, and finally completes the migration of the virtual machines. In the process of implementing the virtual machine migration stage, the virtual machine migration module 300 needs to perform different controls on the cloud server in the dormant state, the cloud server in the low load state, the alarm cloud server, and the cloud server in the normal operation state, and the cloud server classification module 200 completes the classification of the working state of the cloud server.
Specifically, as shown in fig. 8, the cloud server classification module 200 includes the following modules.
The dormant state cloud server classifying module 201: and classifying the cloud server in the dormant state as the dormant state cloud server.
Operating state cloud server classification module 202: and classifying the cloud server in the running state as a running state cloud server.
The operating state cloud server classification module 202 includes the following modules.
The low-load cloud server classification module 20201: and classifying the cloud server with the CPU utilization rate of the cloud server nodes in the running state lower than a set first threshold value into a low-load cloud server.
The alarm cloud server classification module 20202: classifying the cloud server meeting the alarm condition as an alarm cloud server, wherein the meeting of the alarm condition means that any one of the following conditions is met: under a first condition, the utilization rate of a CPU (central processing unit) of a cloud server node is higher than a set second threshold value, wherein the second threshold value is larger than the first threshold value; a second condition, wherein the current predicted temperature of the cloud server is higher than the server alarm temperature;
normal operating state cloud server classification module 20203: and classifying the cloud servers which do not belong to the low-load cloud server and the alarm cloud server in the operation state cloud server list into normal operation state cloud servers, wherein the normal operation state cloud servers can be used as host cloud servers.
As shown in fig. 9, the data acquisition module 110 includes the following modules.
Initial training data set construction module 111: constructing an initial training data set by using historical sampling data; defining the time interval between every two adjacent sampling moments as a sampling period, defining a prediction period on each sampling period, wherein the length of the prediction period is the prediction depth of the temperature prediction model, and the initial time of the prediction period is the initial time of the same sampling period; the method comprises the steps of taking index data of a cloud server at the initial moment of a prediction period as a model input data index, and taking the actually measured cloud server temperature at the end moment of the same prediction period as a model output data index to construct an initial training data set.
The newly added training data set construction module 112: acquiring a newly added training data set; wherein the newly added training data set comprises: index data of the cloud server at the initial time of a prediction period ending at the initial time of the current sampling period is used as a model input data index, and the actually measured cloud server temperature of the cloud server at the end time of the prediction period ending at the initial time of the current sampling period is used as a model output data index; the measured cloud server temperature refers to an arithmetic mean value of temperatures of different parts and/or components of the measured cloud server.
As shown in fig. 4, the cloud server temperature prediction model training data indexes include an input data index and an output data index. The input data indicator includes any one or more of the following: the number of CPU cores of the cloud server; the cloud server uses the CPU core number; the utilization rate of a CPU of the cloud server; memory capacity of a cloud server; the memory utilization rate of the cloud server; current frequency of a cloud server CPU; the cloud server inputs network flow; the cloud server outputs network traffic; the rotating speed of each fan of the cloud server; and the cloud server deploys the number of the virtual machines. The output data index comprises any one of the following data or an arithmetic mean of any plurality of data: actually measuring the inlet air temperature of the cloud server; actually measuring the air outlet temperature of the cloud server; actually measuring the temperature of a CPU of the cloud server; and (4) actually measuring the GPU temperature of the cloud server.
The data preprocessing module 120 includes: initial training data set preprocessing module 121: preprocessing an initial training data set; the newly added training data set preprocessing module 122: and preprocessing the newly added training data set.
The temperature prediction model training module 130 includes the following modules.
Initial training data set training module 131: performing initial training on the Bayesian linear regression model by using the preprocessed initial training data to obtain an initial temperature prediction model; wherein, in the initial training stage of Bayesian linear regression, the prior distribution of model parameters is selected to obey the given initial hyperparameter as,,After initial training of the Bayesian linear regression model is completed, the posterior distribution of model parameters obeys hyper-parameters of,,The gaussian-inverse gama probability distribution of (a), wherein n represents the last sampling period in the initial training dataset; hyper-parameter,,Are all made of,,And simple functions of the initial training sample set, with analytical solutions.
Newly added training data set training module 132: performing online incremental training on the Bayesian linear regression model by using the newly-added training data set after preprocessing in an incremental manner; in the Bayesian linear regression increment training stage, the model parameter before the current sampling period t obeys hyper-parameter of,,In the Gaussian-inverse GAMMA probability distribution, the model parameter of the current sampling period t is updated into the obedience hyper-parameter by adding the sampling training sample of the sampling period t in the current sampling period t,,Gaussian-inverse gama probability distribution; hyper-parameter,,Are all made of,,And a simple function of the incremental training samples, with an analytical solution.
The virtual machine migration module 300 includes the following modules.
Cloud server temperature acquisition module 301: and traversing the cloud server list of the cloud data center to obtain the cloud server temperature predicted value of the end time of the prediction period defined on the sampling period of each cloud server at the current time.
The to-be-migrated virtual machine obtaining module 302: traversing the low-load cloud servers to obtain virtual machines in each low-load cloud server as virtual machines to be migrated, and listing the virtual machines to be migrated in a virtual machine list to be migrated; and traversing the alarm cloud servers to obtain the virtual machine with the highest migration priority in each alarm cloud server as the virtual machine to be migrated, and listing the virtual machine to be migrated in a virtual machine list to be migrated.
The migration operation module 303: and determining a target host cloud server of each virtual machine in the virtual machine list to be migrated according to the predicted temperature and the residual load capacity of each host cloud server, and completing virtual machine migration.
The flow end determination module 304: judging whether the process is finished or not; if yes, ending the process; if not, triggering the cloud server temperature obtaining module 301 to obtain the predicted temperature of the cloud server in the next virtual machine migration planning period.
Specifically, in fig. 9, first, the initial training data set constructing module 111 sends the constructed initial training data set to the initial training data set preprocessing module 121 for preprocessing, the initial training data set preprocessing module 121 sends the preprocessed initial training data set to the initial training data set training module 131 for training the bayesian linear regression model, so as to obtain an initial temperature prediction bayesian linear regression model, and trigger the newly added training data set constructing module 112 to collect newly added training data. Secondly, the newly added training data set constructing module 112 sends the constructed newly added training data set to the newly added training data set preprocessing module 122 for preprocessing, the newly added training data set preprocessing module 122 sends the preprocessed newly added training data set to the newly added training data set training module 132 to perform incremental training on the current bayesian linear regression model, and triggers the temperature predicting module 140 to perform cloud server temperature prediction, so as to obtain a predicted cloud server temperature value. After the prediction process is finished, at the initial time of the next sampling period, the temperature prediction module 140 triggers the newly added training data set construction module 112 to collect newly added training data, and starts the next incremental training prediction implementation period of the cloud server temperature prediction model.
Independently of incremental training of the temperature prediction model of the cloud server by the temperature prediction model training prediction module 100 and prediction of the cloud server temperature, migration of the virtual machine is controlled by the virtual machine migration module 300, wherein at the initial time of a virtual machine migration planning period, the cloud server temperature acquisition module 301 acquires, from the temperature prediction module 140, predicted temperatures at the end time of a prediction period defined on a sampling period where all cloud servers including an operating state cloud server and a dormant state cloud server are located at the current time, and triggers the virtual machine to be migrated acquisition module 302; the virtual machine to be migrated acquiring module 302 determines a virtual machine to be migrated and triggers the migration operation module 303, the migration operation module 303 migrates the virtual machine according to the predicted temperature of the cloud server acquired by the cloud server temperature acquiring module 301 and in combination with the residual load capacity of the cloud server, and then the flow end determining module 304 is triggered to determine whether the flow is ended; if the process is not finished, triggering the cloud server temperature obtaining module 301 to obtain the predicted temperature of the cloud server, and starting the next virtual machine migration planning cycle. The to-be-migrated virtual machine obtaining module 302 and the migration operation module 303 invoke the cloud server allocation module 200 to classify the cloud servers and apply different controls.
More specifically, the sampling period, the prediction period, the monitoring period, and the virtual machine migration planning period are described in more detail with reference to fig. 9. The sampling period is a working period of the temperature prediction model training prediction module 100, wherein the temperature prediction module 140 triggers the newly added training data set construction module 112 to collect newly added data according to the sampling period. The prediction period is a prediction depth of the temperature prediction model, and is greater than the sampling period, and the temperature prediction module 140 performs temperature prediction according to the prediction period defined on each sampling period as the prediction depth. The monitoring period is a work period in which the cloud server classification module 200 monitors the state of the cloud server. The virtual machine migration planning period is a work period for the virtual machine migration module 300 to plan and migrate the virtual machine, the virtual machine migration planning period is based on the time required for scheduling actually, and the monitoring period and the virtual machine migration planning period start and end synchronously. The temperature prediction model training prediction module 100 and the virtual machine migration module 300 operate independently according to a sampling period and a virtual machine migration planning period, respectively, the virtual machine migration planning period is greater than the sampling period, and when the next virtual machine migration planning period comes, the cloud server temperature acquisition module 301 acquires, from the temperature prediction module 140, the predicted temperature at the end of the prediction period defined on the current sampling period.
As shown in fig. 10, the initial training data set preprocessing module 121 and the newly added training data set preprocessing module 122 each include the following modules.
The data cleaning module 1201: if at the sampling instantObtained values of sampled data itemsFor the empty value, the empty value is filled using the historical average of the data item.
The normalization module 1202: for each data item, respectively recording the minimum value and the maximum value of the data item in the initial training data set as、The original value of the data itemMapped to approximate interval [0,1 ]]Is new value ofThe formula is as follows:
the value of the data item obtained after preprocessing each data item in the training sampleThe preprocessed training samples are combined as the input of the initial training and incremental training of the bayesian linear regression in the temperature prediction model training module 130.
Specifically, after the data acquisition module 110 acquires the training data, the data cleaning module 1201 completes data cleaning and triggers the normalization module 1202, and the normalization module 1202 normalizes the training data that has completed data cleaning and sends the obtained normalized data to the temperature prediction model training module 130 for training.
As shown in fig. 11, in the migration operation module 303, a virtual machine migration priority V is calculated by the following module.
The user priority weighting coefficient obtaining module 30201: obtaining a user level according to the monthly consumption amount or the annual consumption amount of the user; obtaining a user priority weighting coefficient R according to the user level; the user priority weighting coefficient R comprises a numerical value between 0 and 1.
The virtual machine constant migration load weight coefficient obtaining module 30202: obtaining the memory size of the virtual machineP(ii) a Calculating the average value of the memory sizes of all virtual machines in the cloud server in the running state(ii) a According to the memory size of the virtual machinePAnd the average value of the memory sizes of all virtual machines in all running state cloud serversCalculating the constant migration load weight coefficient of the virtual machine,。
The virtual machine incremental migration load weight coefficient obtaining module 30203: obtaining the memory change rate of the virtual machine in unit timeS(ii) a Calculating the average value of the memory change rates of all virtual machines in all running state cloud servers in unit time(ii) a According to the unit time memory change rate of the virtual machineSAnd average value of memory change rate of all virtual machines in unit time in running state cloud serverCalculating the load weight coefficient of the incremental migration of the virtual machine,。
The migration priority acquisition module 30204 of the virtual machine: and calculating the migration priority of the virtual machine according to the user priority weight coefficient, the virtual machine constant migration load weight coefficient and the virtual machine incremental migration load weight coefficient.
According to a migration priority calculation formula:wherein, in the step (A),,then virtual machine migration priority
Wherein R represents a user priority weighting factor,representing a virtual machine constant migration load weight coefficient,represents the load weight coefficient of the incremental migration of the virtual machine, P represents the memory size of the virtual machine,represents the average memory size of all the virtual machines in all the running state cloud servers,represents the memory change rate of the virtual machine per unit time,represents the average memory change rate per unit time of all the virtual machines in all the running state cloud servers,the value range (0, 1) is an empirical parameter]。
As shown in fig. 12, the migration operation module 303 includes the following modules.
The migration matching relationship assignment module 3031: traversing the virtual machines to be migrated in the virtual machine list to be migrated, and distributing target host cloud servers to the virtual machines to be migrated;
the migration matching relationship output module 3032: migrating the virtual machine to be migrated according to the matching relationship between the virtual machine to be migrated and the target host cloud server, which is established by the migration matching relationship allocation module 3031;
wherein the migration matching relationship assignment module 3031 includes:
predicted temperature ordering module 303101: sorting the normally-running cloud servers from low to high according to the predicted temperature obtained by the cloud server temperature obtaining module 301 to obtain a sorted list of cloud servers to be selected;
virtual machine to be migrated traverse module 303102: traversing the virtual machine to be migrated in the virtual machine list to be migrated, and acquiring the next virtual machine in the virtual machine list to be migrated as the current virtual machine to be migrated;
the to-be-selected cloud server screening module 303103: according to the order of the list of the cloud servers to be selected after the predicted temperature sorting module 303101 sorts, the cloud servers to be selected which meet the following conditions are allocated to the current virtual machine to be migrated as target host cloud servers:
the predicted temperature is less than the server alarm temperature threshold;
the number of the residual CPU cores is larger than the CPU core number requirement of the virtual machine to be migrated;
the amount of the residual unallocated memory is greater than the memory requirement of the virtual machine to be migrated;
sleep state cloud server wake-up module 303104: for the current to-be-migrated virtual machine to which the target host cloud server is not allocated by the to-be-selected cloud server screening module 303103, listing a cloud server from the dormant state cloud server list, listing the cloud server in the normal operation state cloud server list, and allocating the cloud server as the current to-be-selected cloud server to the to-be-migrated virtual machine.
Specifically, the description will be made with reference to fig. 9 and 12. Under the trigger of the virtual machine to be migrated acquisition module 302, the migration matching relationship allocation module 3031 traverses the virtual machines to be migrated in the virtual machine to be migrated list provided by the virtual machine to be migrated acquisition module 302, allocates target host cloud servers to the virtual machines to be migrated, obtains the matching relationship between the virtual machines to be migrated and the target host cloud servers, and sends the matching relationship to the migration matching relationship output module 3032; the migration matching relationship output module 3032 migrates the virtual machine to be migrated according to the matching relationship between the virtual machine to be migrated and the target host cloud server, and triggers the flow end determination module 304 to continue processing.
More specifically, as shown in fig. 12, for a given virtual machine list to be migrated, the predicted temperature sorting module 303101 sorts the normally running cloud servers from low to high according to the predicted temperature to obtain a sorted cloud server list to be selected, and then triggers the virtual machine to be migrated traversing module 303102; the virtual machine to be migrated traversing module 303102 determines whether the virtual machine in the virtual machine to be migrated list has been traversed; if yes, triggering a migration matching relationship output module 3032 and sending the matching relationship to the migration matching relationship output module 3032; if not, acquiring the next virtual machine in the virtual machine list to be migrated as the current virtual machine to be migrated, and then triggering the cloud server screening module to be selected 303103; the to-be-selected cloud server screening module 303103 allocates, according to the order of the to-be-selected cloud server list sorted by the predicted temperature sorting module 303101, a to-be-selected cloud server that meets the following conditions as a target host cloud server to the current to-be-migrated virtual machine:
the predicted temperature is less than the server alarm temperature threshold;
the number of the residual CPU cores is larger than the CPU core number requirement of the virtual machine to be migrated;
the amount of the residual unallocated memory is greater than the memory requirement of the virtual machine to be migrated;
then, the to-be-selected cloud server screening module 303103 triggers the sleep state cloud server wake-up module 303104. The dormant state cloud server waking module 303104 determines that if the current virtual machine to be migrated has been allocated with the target host cloud server by the to-be-selected cloud server screening module 303103, the dormant state cloud server waking module 303104 triggers the to-be-migrated virtual machine traversal module 303102 to continue processing on the next to-be-migrated virtual machine; if the current virtual machine to be migrated is not allocated with the target host cloud server by the to-be-selected cloud server screening module 303103, the dormant state cloud server wakeup module 303104 lists a cloud server from the dormant state cloud server list, lists the cloud server in the normal operation state cloud server list, allocates the cloud server as the current cloud server to be selected to the current virtual machine to be migrated, and then triggers the to-be-migrated virtual machine traversal module 303102 to continue processing on the next virtual machine to be migrated.
Third embodiment
According to the present invention, there is provided a computer readable storage medium storing a computer program, which when executed by a processor implements the steps of the virtual machine migration planning and scheduling method based on temperature prediction.
Those skilled in the art will appreciate that, in addition to implementing the system and its various modules provided by the present invention in purely computer readable program code means, the same procedures can be implemented entirely by logically programming method steps such that the system and its various modules are provided in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Therefore, the system and the modules thereof provided by the present invention can be considered as a hardware component, and the modules included in the system and used for implementing various programs can also be considered as structures in the hardware component; modules for performing various functions may also be considered to be both software programs for performing the methods and structures within hardware components.
The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes or modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention. The embodiments and features of the embodiments of the present application may be combined with each other arbitrarily without conflict.
Claims (18)
1. A virtual machine migration planning and scheduling method based on temperature prediction is characterized by comprising the following steps:
step S1: the method comprises the steps of obtaining a cloud server list of a cloud data center, obtaining a predicted value of a Bayesian linear regression-based temperature prediction model for the temperature of each cloud server in the cloud server list of the cloud data center, and obtaining the predicted temperature of the cloud server;
step S2: defining the operating state cloud server with the load monitoring index meeting the low-load condition in the operating state cloud server list as a low-load cloud server, and constructing a low-load cloud server list;
step S3: traversing the low-load cloud server list, acquiring the virtual machines in each low-load cloud server, and listing the virtual machines into a to-be-migrated virtual machine list;
step S4: determining a target host cloud server of each virtual machine in the virtual machine list to be migrated according to the predicted temperature and the residual load capacity of each host cloud server in the host cloud server list, and completing virtual machine migration; the host cloud server comprises a normal operation state cloud server and cloud servers in a dormant state cloud server list; the selection sequence of the target host cloud server is that the cloud server in the normal operation state is preferentially selected, and when the cloud server in the normal operation state does not meet the matching condition, the cloud server in the dormant state is selected; the cloud server in the normal operation state is defined as a cloud server which is not listed in the low-load cloud server list and is not listed in the alarm cloud server list in the operation state cloud server list;
step S5: setting all the migrated low-load cloud servers of the virtual machines into a dormant state, listing an operating state cloud server list, and adding the operating state cloud server list into the dormant state cloud server list;
step S6: defining the operating state cloud server with the monitoring index meeting the alarm condition in the operating state cloud server list as an alarm cloud server, and listing all alarm cloud servers in an alarm cloud server list;
step S7: traversing the alarm cloud server list to obtain the virtual machine with the highest migration priority in each alarm cloud server, and listing the virtual machines into a to-be-migrated virtual machine list;
step S8: determining a target host cloud server of each virtual machine in the virtual machine list to be migrated according to the predicted temperature and the residual load capacity of each host cloud server in the host cloud server list, and completing virtual machine migration;
step S9: setting alarm cloud servers, from which the virtual machines are all migrated, to be in a dormant state, listing an operating state cloud server list, and adding the operating state cloud server list into the dormant state cloud server list;
step S10: judging whether the process is finished or not; if yes, ending the process; if not, the process goes to the step S1 to continue.
2. The method for scheduling virtual machine migration planning based on temperature prediction according to claim 1, wherein the temperature prediction model in step S1 is obtained through the following steps:
step S101: establishing an initial training data set by using historical sampling data, and performing initial training on a Bayesian linear regression model to obtain an initial temperature prediction model;
step S102: in the daily operation process of the cloud data center, acquiring a newly added training data set in real time, performing online incremental training on a current cloud server temperature prediction model in an incremental mode, updating the temperature prediction model, and predicting the future temperature of the cloud server by using the temperature prediction model to obtain a predicted value of the future temperature of the cloud server;
wherein:
the step S101 includes the steps of:
step S10101: defining the time interval between every two adjacent sampling moments as a sampling period, defining a prediction period on each sampling period, wherein the length of the prediction period is the prediction depth of the temperature prediction model, and the initial time of the prediction period is the initial time of the same sampling period; the method comprises the steps that index data of a cloud server at the initial moment of a prediction period are used as model input data indexes, and the actually measured cloud server temperature at the end moment of the same prediction period is used as model output data indexes to construct an initial training data set; the measured cloud server temperature is defined as an arithmetic mean value of temperatures of different parts and/or components of the measured cloud server;
step S10102: preprocessing an initial training data set;
step S10103: training the Bayes linear regression model by utilizing the preprocessed initial training data set to obtain a parameter posterior distribution function of the initial Bayes linear regression model;
step S10104: outputting a parameter posterior distribution function of the initial Bayes linear regression model;
the step S102 includes the steps of:
step S10201: inputting a parameter posterior distribution function of a current Bayes linear regression model;
step S10202: acquiring a newly added training data set; wherein the newly added training data set comprises: index data of the cloud server at the initial time of a prediction period ending at the initial time of the current sampling period is used as an input data index, and the actually measured cloud server temperature of the cloud server at the end time of the prediction period ending at the initial time of the current sampling period is used as an output data index; the measured cloud server temperature is defined as an arithmetic mean value of temperatures of different parts and/or components of the measured cloud server;
step S10203: preprocessing the newly added training data set;
step S10204: performing incremental training on the current Bayes linear regression model by using the newly-added training data set after preprocessing, and updating a parameter posterior distribution function of the current Bayes linear regression model;
step S10205: obtaining a Bayesian optimal estimation of a parameter of a current Bayesian linear regression model;
step S10206: predicting the temperature of the cloud server at the end moment of the current prediction period based on the parameter Bayes optimal estimation of the current Bayes linear regression model;
step S10207: outputting the predicted temperature of the cloud server;
step S10208: judging whether the predicted temperature process of the cloud server is finished or not; if yes, ending the process; if not, the process goes to step S10202 to continue the process.
3. The virtual machine migration planning and scheduling method based on temperature prediction according to claim 2, wherein the cloud server temperature prediction model training data indexes in steps S10101 and S10202 include an input data index and an output data index;
the input data indicator includes any one or more of the following:
the number of CPU cores of the cloud server;
the cloud server uses the CPU core number;
the utilization rate of a CPU of the cloud server;
memory capacity of a cloud server;
the memory utilization rate of the cloud server;
current frequency of a cloud server CPU;
the cloud server inputs network flow;
the cloud server outputs network traffic;
the rotating speed of each fan of the cloud server;
the number of the virtual machines deployed by the cloud server nodes is increased;
the output data index comprises any one of the following data or an arithmetic mean of any plurality of data:
actually measuring the inlet air temperature of the cloud server;
actually measuring the air outlet temperature of the cloud server;
actually measuring the temperature of a CPU of the cloud server;
and (4) actually measuring the GPU temperature of the cloud server.
4. The method for scheduling virtual machine migration planning based on temperature prediction according to claim 2, wherein the preprocessing in step S10102 and step S10203 includes the following steps:
a data cleaning step: if at the sampling instantObtained values of sampled input data itemsFor the vacancy value, filling the vacancy value with the historical average of the data item;
a normalization step: for each input data item, the minimum value and the maximum value of the data item in the initial training data set defined in the step S10102 are recorded as the minimum value and the maximum value respectively、Inputting the original value of the data itemMapped to approximate interval [0,1 ]]Is new value ofThe formula is as follows:
each input in the training sampleValues of data items obtained after preprocessing of the data itemsAnd merging the preprocessed training sample input data as the input of the initial Bayesian linear regression training in the step S10103 and the incremental Bayesian linear regression training in the step S10204.
5. The method according to claim 2, wherein the virtual machine migration plan scheduling method based on temperature prediction,
the step S10103 includes: in the initial training stage of Bayesian linear regression, the prior distribution of model parameters is selected to obey the given initial hyperparameter as,,After initial training of the Bayesian linear regression model is completed, the posterior distribution of model parameters obeys hyper-parameters of,,The gaussian-inverse gama probability distribution of (a), wherein n represents the last sampling period in the initial training dataset; hyper-parameter,,Are all made of,,And a simple function of the initial training sample set, with an analytic solution;
the step S10204 includes: in the Bayes linear regression increment training stage, the model parameter before the current sampling period t obeys hyper-parameter of,,In the Gaussian-inverse GAMMA probability distribution, the model parameter of the current sampling period t is updated into the obedience hyper-parameter by adding the sampling training sample of the sampling period t in the current sampling period t,,Gaussian-inverse gama probability distribution; hyper-parameter,,Are all made of,,And a simple function of the incremental training samples, with an analytical solution.
6. The virtual machine migration planning and scheduling method based on temperature prediction of claim 1, wherein the low load condition is: the CPU utilization rate of the cloud server in the running state is lower than a set first threshold value;
the alarm condition is satisfied, that is, any one of the following conditions is satisfied:
under a first condition, the utilization rate of a CPU (central processing unit) of the cloud server is higher than a set second threshold value, wherein the second threshold value is larger than the first threshold value;
in a second condition, the current predicted temperature of the cloud server is higher than the server alarm temperature.
7. The method according to claim 1, wherein in step S8, the virtual machine migration priority V is calculated according to the following steps:
step S801: obtaining a user level according to the monthly consumption amount or the annual consumption amount of the user; obtaining a user priority weight coefficient according to the user level(ii) a The user priority weighting factorIncluding values between 0 and 1;
step S802: obtaining the memory size of a virtual machine(ii) a Calculating the average value of the memory sizes of all virtual machines in the cloud server in the running state(ii) a According to the memory size of the virtual machineAnd the average value of the memory sizes of all virtual machines in all running state cloud serversCalculating the constant migration load weight coefficient of the virtual machine,;
Step S803: obtaining the memory change rate of the virtual machine in unit time(ii) a Calculating the average value of the memory change rates of all virtual machines in all running state cloud servers in unit time(ii) a According to the unit time memory change rate of the virtual machineAnd average value of memory change rate of all virtual machines in unit time in running state cloud serverCalculating the load weight coefficient of incremental migration of the virtual machine,;
Step S804: calculating the migration priority of the virtual machine according to the user priority weight coefficient, the virtual machine constant migration load weight coefficient and the virtual machine incremental migration load weight coefficient:
according to a migration priority calculation formula:wherein, in the step (A),,then virtual machine migration priority
Wherein the content of the first and second substances,represents a user priority weight coefficient and,representing a virtual machine constant migration load weight coefficient,representing a virtual machine incremental migration load weight coefficient,which represents the size of the memory of the virtual machine,represents the average memory size of all the virtual machines in all the running state cloud servers,represents the memory change rate of the virtual machine per unit time,represents the average memory change rate per unit time of all the virtual machines in all the running state cloud servers,the value range (0, 1) is an empirical parameter]。
8. The virtual machine migration planning and scheduling method based on temperature prediction according to claim 1, wherein in the step S4 and the step S8, the step of determining the target host cloud server of each virtual machine in the virtual machine list to be migrated according to the predicted temperature and the residual load capacity of each host cloud server in the host cloud server list all uses a minimum temperature server allocation principle algorithm, which specifically includes the following steps:
step A: traversing the virtual machines to be migrated in the virtual machine list to be migrated, and executing a first loop step for each virtual machine to be migrated, wherein the first loop step comprises the following steps:
step a: traversing all the cloud servers in the normal operation state, and acquiring the current preset temperature of each cloud server in the normal operation state;
step b: sorting the normally-operated cloud servers from low to high according to the predicted temperature to obtain a sorted list of cloud servers to be selected;
step c: traversing the sorted cloud server list to be selected, and executing a second circulation step, wherein the second circulation step comprises the following steps:
step c 1: for the current cloud server to be selected, judging whether the current cloud server to be selected meets the following conditions:
the predicted temperature is less than the server alarm temperature threshold;
the number of the residual CPU cores is larger than the CPU core number requirement of the virtual machine to be migrated;
the amount of the residual unallocated memory is greater than the memory requirement of the virtual machine to be migrated;
if so, distributing the virtual machine to be migrated to the current cloud server to be selected to obtain a target host cloud server; exiting the second loop step;
if not, go to step c2 to continue execution;
step c 2: judging whether a cloud server list to be selected has a next cloud server to be selected; if yes, go to step c3 to continue execution; if not, jumping to the step c4 to continue execution;
step c 3: taking the next cloud server to be selected as the current cloud server to be selected, and jumping to the step c1 to continue execution;
step c 4: listing a cloud server from the dormant state cloud server list, listing the cloud server into the normal operation state cloud server list, taking the cloud server as the current cloud server to be selected, and jumping to the step c1 to continue execution;
and B: judging whether a next virtual machine to be migrated exists or not; if not, entering the step C to continue execution; if yes, entering the step A to continue executing the first circulation step;
and C: and obtaining the matching relation between the virtual machine to be migrated and the target host cloud server.
9. A virtual machine migration planning and scheduling system based on temperature prediction is characterized by comprising:
temperature prediction model training prediction module (100): training a temperature prediction model based on Bayesian linear regression, and predicting the temperature by using the temperature prediction model to obtain the predicted temperature of the cloud server;
cloud server classification module (200): classifying the cloud servers into an operating state cloud server and a dormant state cloud server according to the working state, wherein the operating state cloud server is classified into a low-load cloud server, a normal operating state cloud server and an alarm state cloud server;
virtual machine migration module (300): based on the classification of the cloud server by the cloud server classification module (200), according to the predicted temperature of the cloud server obtained by the temperature prediction model training prediction module (100), migrating the virtual machine on the low-load cloud server and the virtual machine with the highest migration priority on the alarm cloud server to the host cloud server; the host cloud server comprises a normal operation state cloud server and cloud servers in a dormant state cloud server list; the selection sequence of the target host cloud server is that the cloud server in the normal operation state is preferentially selected, and when the cloud server in the normal operation state does not meet the matching condition, the cloud server in the dormant state is selected; the cloud server in the normal operation state is defined as a cloud server which is not listed in the low-load cloud server list and is not listed in the alarm cloud server list in the operation state cloud server list.
10. The virtual machine migration planning scheduling system based on temperature prediction according to claim 9, wherein the temperature prediction model training prediction module (100) comprises:
data acquisition module (110): acquiring index data of a cloud server and actually measured cloud server temperature data;
a data pre-processing module (120): preprocessing data acquired by a data acquisition module (110);
a temperature prediction model training module (130): training a Bayesian linear regression model by using data preprocessed by a data preprocessing module (120);
temperature prediction module (140): and predicting the temperature according to the current Bayes linear regression model trained by the temperature prediction model training module (130) to obtain the predicted temperature of the cloud server.
11. The virtual machine migration planning scheduling system based on temperature prediction according to claim 10, wherein the data collection module (110) comprises:
an initial training data set construction module (111): constructing an initial training data set by using historical sampling data; defining the time interval between every two adjacent sampling moments as a sampling period, defining a prediction period on each sampling period, wherein the length of the prediction period is the prediction depth of the temperature prediction model, and the initial time of the prediction period is the initial time of the same sampling period; the method comprises the steps that index data of a cloud server at the initial moment of a prediction period are used as model input data indexes, and the actually measured cloud server temperature at the end moment of the same prediction period is used as model output data indexes to construct an initial training data set; the measured cloud server temperature is defined as an arithmetic mean value of temperatures of different parts and/or components of the measured cloud server;
a newly added training data set construction module (112): acquiring a newly added training data set; wherein the newly added training data set comprises: index data of the cloud server at the initial time of a prediction period ending at the initial time of the current sampling period is used as a model input data index, and the actually measured cloud server temperature of the cloud server at the end time of the prediction period ending at the initial time of the current sampling period is used as a model output data index; the measured cloud server temperature is defined as an arithmetic mean value of temperatures of different parts and/or components of the measured cloud server;
the cloud server temperature prediction model training data indexes comprise input data indexes and output data indexes;
the input data indicator includes any one or more of the following:
the number of CPU cores of the cloud server;
the cloud server uses the CPU core number;
the utilization rate of a CPU of the cloud server;
memory capacity of a cloud server;
the memory utilization rate of the cloud server;
current frequency of a cloud server CPU;
the cloud server inputs network flow;
the cloud server outputs network traffic;
the rotating speed of each fan of the cloud server;
the number of the virtual machines deployed by the cloud server is increased;
the output data index comprises any one of the following data or an arithmetic mean of any plurality of data:
actually measuring the inlet air temperature of the cloud server;
actually measuring the air outlet temperature of the cloud server;
actually measuring the temperature of a CPU of the cloud server;
and (4) actually measuring the GPU temperature of the cloud server.
12. The virtual machine migration planning scheduling system based on temperature prediction according to claim 10, wherein the data preprocessing module (120) comprises:
an initial training data set preprocessing module (121): preprocessing an initial training data set;
a newly added training data set preprocessing module (122): preprocessing the newly added training data set;
the initial training data set preprocessing module (121) and the newly added training data set preprocessing module (122) both comprise:
data cleansing module (1201): if at the sampling instantObtained values of sampled data itemsFor empty values, fill-in with historical averages of the data itemsA vacancy value;
normalization module (1202): for each data item, respectively recording the minimum value and the maximum value of the data item in the initial training data set as、The original value of the data itemMapped to approximate interval [0,1 ]]Is new value ofThe formula is as follows:
the value of the data item obtained after preprocessing each data item in the training sampleThe preprocessed training samples are combined to be used as the input of Bayesian linear regression initial training and Bayesian linear regression increment training in the temperature prediction model training module (130).
13. The virtual machine migration planning scheduling system based on temperature prediction of claim 10, wherein the temperature prediction model training module (130) comprises:
an initial training data set training module (131): performing initial training on the Bayesian linear regression model by using the preprocessed initial training data to obtain an initial temperature prediction model; wherein, in the initial training stage of Bayesian linear regression, the prior distribution of model parameters is selected to obey the given initial hyperparameter as,,After initial training of the Bayesian linear regression model is completed, the posterior distribution of model parameters obeys hyper-parameters of,,The gaussian-inverse gama probability distribution of (a), wherein n represents the last sampling period in the initial training dataset; hyper-parameter,,Are all made of,,And a simple function of the initial training sample set, with an analytic solution;
a newly added training data set training module (132): performing online incremental training on the Bayesian linear regression model by using the newly-added training data set after preprocessing in an incremental manner; in the Bayesian linear regression increment training stage, the model parameter before the current sampling period t obeys hyper-parameter of,,In the Gaussian-inverse GAMMA probability distribution, the model parameter of the current sampling period t is updated into the obedience hyper-parameter by adding the sampling training sample of the sampling period t in the current sampling period t,,Gaussian-inverse gama probability distribution; hyper-parameter,,Are all made of,,And a simple function of the incremental training samples, with an analytical solution.
14. The virtual machine migration planning scheduling system based on temperature prediction according to claim 9, wherein the cloud server classification module (200) comprises:
dormant state cloud server classification module (201): classifying the cloud server in the dormant state as a dormant state cloud server; the dormant cloud server can be used as a host cloud server;
operating state cloud server classification module (202): classifying the cloud server in the running state into a cloud server in the running state;
the operating state cloud server classification module (202) comprises:
low-load cloud server classification module (20201): classifying cloud servers with the operating state cloud server node CPU utilization rate lower than a set first threshold value into low-load cloud servers;
alarm cloud server classification module (20202): classifying the cloud server meeting the alarm condition as an alarm cloud server, wherein the meeting of the alarm condition means that any one of the following conditions is met:
under a first condition, the utilization rate of a CPU (central processing unit) of a cloud server node is higher than a set second threshold value, wherein the second threshold value is larger than the first threshold value;
a second condition, wherein the current predicted temperature of the cloud server is higher than the server alarm temperature;
normal operating state cloud server classification module (20203): and classifying the cloud servers which do not belong to the low-load cloud server and the alarm cloud server in the operation state cloud server list into normal operation state cloud servers, wherein the normal operation state cloud servers can be used as host cloud servers.
15. The virtual machine migration planning scheduling system based on temperature prediction according to claim 9, wherein the virtual machine migration module (300) comprises:
cloud server temperature acquisition module (301): traversing a cloud server list of a cloud data center, and acquiring a cloud server temperature predicted value at the end time of a prediction period defined on a sampling period of each cloud server at the current time;
a virtual machine to be migrated acquisition module (302): traversing the low-load cloud servers, obtaining the virtual machines in each low-load cloud server as virtual machines to be migrated, and listing the virtual machines to be migrated in a virtual machine list to be migrated; traversing the alarm cloud servers to obtain a virtual machine with the highest migration priority in each alarm cloud server as a virtual machine to be migrated, and listing the virtual machine to be migrated in a virtual machine list to be migrated;
migration operation module (303): determining a target host cloud server of each virtual machine in the virtual machine list to be migrated according to the predicted temperature and the residual load capacity of each host cloud server, and completing virtual machine migration;
a flow end judgment module (304): judging whether the process is finished or not; if yes, ending the process; if not, triggering the cloud server temperature acquisition module (301) to enter the next virtual machine migration planning period, and continuing to acquire the cloud server predicted temperature.
16. The virtual machine migration planning scheduling system based on temperature prediction according to claim 15, wherein in the to-be-migrated virtual machine obtaining module (302), the virtual machine migration priority V is calculated by:
a user priority weight coefficient acquisition module (30201): obtaining a user level according to the monthly consumption amount or the annual consumption amount of the user; obtaining a user priority weighting coefficient R according to the user level; the user priority weighting coefficient R comprises a numerical value between 0 and 1;
a virtual machine constant migration load weight coefficient acquisition module (30202): obtaining the memory size of a virtual machineP(ii) a Computing all virtual machines in running state cloud serverAverage of memory size(ii) a According to the memory size of the virtual machinePAnd the average value of the memory sizes of all virtual machines in all running state cloud serversCalculating the constant migration load weight coefficient of the virtual machine,;
A virtual machine incremental migration load weight coefficient acquisition module (30203): obtaining the memory change rate of the virtual machine in unit timeS(ii) a Calculating the average value of the memory change rates of all virtual machines in all running state cloud servers in unit time(ii) a According to the unit time memory change rate of the virtual machineSAnd average value of memory change rate of all virtual machines in unit time in running state cloud serverCalculating the load weight coefficient of incremental migration of the virtual machine,;
A migration priority acquisition module (30204) of the virtual machine: calculating the migration priority of the virtual machine according to the user priority weight coefficient, the virtual machine constant migration load weight coefficient and the virtual machine incremental migration load weight coefficient:
according to a migration priority calculation formula:wherein, in the step (A),,then virtual machine migration priority
Wherein R represents a user priority weighting factor,representing a virtual machine constant migration load weight coefficient,represents the load weight coefficient of the incremental migration of the virtual machine, P represents the memory size of the virtual machine,represents the average memory size of all the virtual machines in all the running state cloud servers,represents the memory change rate of the virtual machine per unit time,represents the average memory change rate per unit time of all the virtual machines in all the running state cloud servers,is a warpThe parameter is tested, and the value range is (0, 1)]。
17. The virtual machine migration planning scheduling system based on temperature prediction of claim 15 wherein the migration operation module (303) comprises:
a migration matching relationship assignment module (3031): traversing the virtual machines to be migrated in the virtual machine list to be migrated, and distributing target host cloud servers to the virtual machines to be migrated;
a migration matching relationship output module (3032): migrating the virtual machine to be migrated according to the matching relationship between the virtual machine to be migrated and the target host cloud server, which is established by the migration matching relationship allocation module (3031);
wherein the migration matching relationship assignment module (3031) comprises:
predicted temperature ranking module (303101): sorting the normally-running cloud servers from low to high according to the predicted temperature obtained by the cloud server temperature obtaining module (301) to obtain a sorted list of cloud servers to be selected;
virtual machine to be migrated traverse module (303102): traversing the virtual machine to be migrated in the virtual machine list to be migrated, and acquiring the next virtual machine in the virtual machine list to be migrated as the current virtual machine to be migrated;
a to-be-selected cloud server screening module (303103): according to the sequence of the list of the cloud servers to be selected after the predicted temperature sorting module (303101) sorts the list, allocating the cloud servers to be selected, which meet the following conditions, as target host cloud servers to the current virtual machine to be migrated:
the predicted temperature is less than the server alarm temperature threshold;
the number of the residual CPU cores is larger than the CPU core number requirement of the virtual machine to be migrated;
the amount of the residual unallocated memory is greater than the memory requirement of the virtual machine to be migrated;
dormant state cloud server wake-up module (303104): for the current virtual machine to be migrated, to which the target host cloud server is not allocated by the to-be-selected cloud server screening module (303103), listing a cloud server from the dormant state cloud server list, listing the cloud server in the normal operation state cloud server list, and allocating the cloud server as the current cloud server to be selected to the virtual machine to be migrated.
18. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the method for scheduling virtual machine migration planning based on temperature prediction according to any of claims 1 to 8.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202011296139.7A CN112395046B (en) | 2020-07-30 | 2020-07-30 | Virtual machine migration planning and scheduling method, system and medium thereof |
CN202010748491.3A CN111625321B (en) | 2020-07-30 | 2020-07-30 | Virtual machine migration planning and scheduling method based on temperature prediction, system and medium thereof |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010748491.3A CN111625321B (en) | 2020-07-30 | 2020-07-30 | Virtual machine migration planning and scheduling method based on temperature prediction, system and medium thereof |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202011296139.7A Division CN112395046B (en) | 2020-07-30 | 2020-07-30 | Virtual machine migration planning and scheduling method, system and medium thereof |
Publications (2)
Publication Number | Publication Date |
---|---|
CN111625321A true CN111625321A (en) | 2020-09-04 |
CN111625321B CN111625321B (en) | 2020-10-23 |
Family
ID=72258622
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202011296139.7A Active CN112395046B (en) | 2020-07-30 | 2020-07-30 | Virtual machine migration planning and scheduling method, system and medium thereof |
CN202010748491.3A Active CN111625321B (en) | 2020-07-30 | 2020-07-30 | Virtual machine migration planning and scheduling method based on temperature prediction, system and medium thereof |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202011296139.7A Active CN112395046B (en) | 2020-07-30 | 2020-07-30 | Virtual machine migration planning and scheduling method, system and medium thereof |
Country Status (1)
Country | Link |
---|---|
CN (2) | CN112395046B (en) |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112036504A (en) * | 2020-09-15 | 2020-12-04 | 平安国际智慧城市科技股份有限公司 | Temperature measurement model training method, device, equipment and storage medium |
CN112231066A (en) * | 2020-10-29 | 2021-01-15 | 北京思特奇信息技术股份有限公司 | JVM (Java virtual machine) memory use-based optimization processing method and system |
CN112379974A (en) * | 2020-12-04 | 2021-02-19 | 海光信息技术股份有限公司 | Virtual machine live migration system and live migration method thereof |
CN113342462A (en) * | 2021-06-02 | 2021-09-03 | 燕山大学 | Cloud computing optimization method, system and medium integrating limitation periodic quasi-dormancy |
CN115167641A (en) * | 2022-07-07 | 2022-10-11 | 凯德技术长沙股份有限公司 | Inertia self-adaptive temperature control method and system |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20220300324A1 (en) * | 2021-03-19 | 2022-09-22 | Mediatek Inc. | Thermal-aware task scheduling |
CN113534936B (en) * | 2021-09-14 | 2021-12-17 | 苏州浪潮智能科技有限公司 | Server fan rotating speed control method, device, equipment and medium |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102096461A (en) * | 2011-01-13 | 2011-06-15 | 浙江大学 | Energy-saving method of cloud data center based on virtual machine migration and load perception integration |
WO2014000779A1 (en) * | 2012-06-27 | 2014-01-03 | Qatar Foundation | An arrangement configured to allocate resources of a plurality of data storage media to a plurality virtual machines and associated method |
CN103984588A (en) * | 2014-02-21 | 2014-08-13 | 中国科学院计算技术研究所 | Virtual machine migration method based on temperature and system thereof |
CN104035542A (en) * | 2014-05-28 | 2014-09-10 | 中国科学院计算技术研究所 | Virtual machine migration method and system for balancing calculation energy consumption and refrigeration energy consumption |
CN106972899A (en) * | 2017-05-11 | 2017-07-21 | 同济大学 | A kind of cooperative frequency spectrum sensing method excavated based on multi-user's history perception data |
Family Cites Families (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP6288275B2 (en) * | 2014-07-22 | 2018-03-07 | 日本電気株式会社 | Virtualization infrastructure management apparatus, virtualization infrastructure management system, virtualization infrastructure management method, and virtualization infrastructure management program |
US9626227B2 (en) * | 2015-03-27 | 2017-04-18 | Intel Corporation | Technologies for offloading and on-loading data for processor/coprocessor arrangements |
CN108092836A (en) * | 2016-11-21 | 2018-05-29 | 深圳市蓝希领地科技有限公司 | The monitoring method and device of a kind of server |
CN106598733A (en) * | 2016-12-08 | 2017-04-26 | 南京航空航天大学 | Three-dimensional virtual resource scheduling method of cloud computing energy consumption key |
US10635475B2 (en) * | 2017-05-11 | 2020-04-28 | International Business Machines Corporation | Migration-adjusted problem ticket analysis and consolidation |
CN107748693A (en) * | 2017-11-30 | 2018-03-02 | 成都启力慧源科技有限公司 | Group's virtual machine scheduling policy under cloud computing environment |
CN109992350B (en) * | 2017-12-29 | 2023-12-01 | 北京华胜天成科技股份有限公司 | Migration method and device for virtual machines in resource pool of cloud computing system |
CN110362388B (en) * | 2018-04-11 | 2021-08-31 | 中移(苏州)软件技术有限公司 | Resource scheduling method and device |
US10776149B2 (en) * | 2018-07-25 | 2020-09-15 | Vmware, Inc. | Methods and apparatus to adjust energy requirements in a data center |
CN109375994B (en) * | 2018-09-10 | 2021-07-13 | 西安交通大学 | Data center task temperature prediction and scheduling method based on RBF neural network |
CN109697108A (en) * | 2018-12-19 | 2019-04-30 | 国云科技股份有限公司 | A kind of multiple data centers dispatching method based on Cost Model |
CN110333991B (en) * | 2019-05-30 | 2022-11-25 | 武汉科技大学 | Method for predicting maximum resource utilization rate of cloud platform tasks |
CN110908783A (en) * | 2019-11-09 | 2020-03-24 | 苏州浪潮智能科技有限公司 | Management and control method, system and equipment for virtual machine of cloud data center |
CN111124602A (en) * | 2019-11-29 | 2020-05-08 | 苏州浪潮智能科技有限公司 | BMC temperature-based virtual machine remote control method and device |
-
2020
- 2020-07-30 CN CN202011296139.7A patent/CN112395046B/en active Active
- 2020-07-30 CN CN202010748491.3A patent/CN111625321B/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102096461A (en) * | 2011-01-13 | 2011-06-15 | 浙江大学 | Energy-saving method of cloud data center based on virtual machine migration and load perception integration |
WO2014000779A1 (en) * | 2012-06-27 | 2014-01-03 | Qatar Foundation | An arrangement configured to allocate resources of a plurality of data storage media to a plurality virtual machines and associated method |
CN103984588A (en) * | 2014-02-21 | 2014-08-13 | 中国科学院计算技术研究所 | Virtual machine migration method based on temperature and system thereof |
CN104035542A (en) * | 2014-05-28 | 2014-09-10 | 中国科学院计算技术研究所 | Virtual machine migration method and system for balancing calculation energy consumption and refrigeration energy consumption |
CN106972899A (en) * | 2017-05-11 | 2017-07-21 | 同济大学 | A kind of cooperative frequency spectrum sensing method excavated based on multi-user's history perception data |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112036504A (en) * | 2020-09-15 | 2020-12-04 | 平安国际智慧城市科技股份有限公司 | Temperature measurement model training method, device, equipment and storage medium |
CN112231066A (en) * | 2020-10-29 | 2021-01-15 | 北京思特奇信息技术股份有限公司 | JVM (Java virtual machine) memory use-based optimization processing method and system |
CN112231066B (en) * | 2020-10-29 | 2024-02-13 | 北京思特奇信息技术股份有限公司 | Optimization processing method and system based on JVM memory use |
CN112379974A (en) * | 2020-12-04 | 2021-02-19 | 海光信息技术股份有限公司 | Virtual machine live migration system and live migration method thereof |
CN113342462A (en) * | 2021-06-02 | 2021-09-03 | 燕山大学 | Cloud computing optimization method, system and medium integrating limitation periodic quasi-dormancy |
CN113342462B (en) * | 2021-06-02 | 2022-03-15 | 燕山大学 | Cloud computing optimization method, system and medium integrating limitation periodic quasi-dormancy |
CN115167641A (en) * | 2022-07-07 | 2022-10-11 | 凯德技术长沙股份有限公司 | Inertia self-adaptive temperature control method and system |
CN115167641B (en) * | 2022-07-07 | 2023-06-20 | 凯德技术长沙股份有限公司 | Inertial self-adaptive temperature control method and system |
Also Published As
Publication number | Publication date |
---|---|
CN112395046A (en) | 2021-02-23 |
CN112395046B (en) | 2021-06-04 |
CN111625321B (en) | 2020-10-23 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111625321B (en) | Virtual machine migration planning and scheduling method based on temperature prediction, system and medium thereof | |
CN110096349B (en) | Job scheduling method based on cluster node load state prediction | |
Hu et al. | Spear: Optimized dependency-aware task scheduling with deep reinforcement learning | |
Gagliolo et al. | Learning dynamic algorithm portfolios | |
CN103607459B (en) | The dynamic resource monitoring of a kind of cloud computing platform IaaS layer and dispatching method | |
US10761897B2 (en) | Predictive model-based intelligent system for automatically scaling and managing provisioned computing resources | |
CN103729248B (en) | A kind of method and apparatus of determination based on cache perception task to be migrated | |
CN104317658A (en) | MapReduce based load self-adaptive task scheduling method | |
CN110389820B (en) | Private cloud task scheduling method for resource prediction based on v-TGRU model | |
JP2017228086A (en) | Machine learning management program, machine learning management method, and machine learning management device | |
WO2024087512A1 (en) | Graph neural network compression method and apparatus, and electronic device and storage medium | |
CN105607952B (en) | Method and device for scheduling virtualized resources | |
CN106528266A (en) | Resource dynamic adjustment method and device in cloud computing system | |
Wang et al. | Energy utilization task scheduling for mapreduce in heterogeneous clusters | |
Huang et al. | Novel heuristic speculative execution strategies in heterogeneous distributed environments | |
CN115168027A (en) | Calculation power resource measurement method based on deep reinforcement learning | |
CN112905343A (en) | Resource scheduling system based on load characteristics in industrial cloud environment | |
CN109117269A (en) | A kind of distributed system dispatching method of virtual machine, device and readable storage medium storing program for executing | |
Dogani et al. | K-agrued: a container autoscaling technique for cloud-based web applications in kubernetes using attention-based gru encoder-decoder | |
CA2637987A1 (en) | Method for autonomic system management using adaptive allocation of resources | |
CN116501505B (en) | Method, device, equipment and medium for generating data stream of load task | |
CN113205128A (en) | Distributed deep learning performance guarantee method based on serverless computing | |
Ismaeel et al. | Real-time energy-conserving vm-provisioning framework for cloud-data centers | |
Serrano-Ruiz et al. | Job shop smart manufacturing scheduling by deep reinforcement learning | |
CN115934349A (en) | Resource scheduling method, device, equipment and computer readable storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |