CN103024048B - Resource regulating method under a kind of cloud environment - Google Patents

Abstract

The invention discloses the resource regulating method under a kind of cloud environment, belong to Distributed Calculation and applications of computer network field.The present invention, from system-level Energy Angle, under the prerequisite of the time of implementation and expense two QoS demand that meet cloud task, dispatches resource for target so that system total energy consumption is minimum, effectively can reduce the energy resource consumption of cloud data center; And adopt Revised genetic algorithum to carry out the search of optimum resource scheduling scheme further, the moderate individuality of fitness is improved to keep population diversity by the selection opertor introducing normal distribution, utilize normal distribution model according to fitness and evolutionary generation design crossover operator and mutation operator to keep population diversity and population structure, thus effectively improve convergence of algorithm performance, decrease the algorithm deadline.

Description

Resource regulating method under a kind of cloud environment

Technical field

The present invention relates to the resource regulating method under a kind of cloud environment, belong to Distributed Calculation and applications of computer network field.

Background technology

Along with user constantly increases the demand of cloud computing, the information technoloy equipment in cloud gets more and more, and data center's scale is increasing, and the energy consumption problem under cloud environment is also more and more outstanding.According to Environmental Protection Agency (US EPA) report, within 2006, U.S. Data center consumes the electricity up to 61,000,000,000 kilowatts altogether.According to estimates, the annual power consumption of data center having 50000 computing nodes is more than 100,000,000 kilowatt hours, and the electricity charge reach 9,300,000 dollars.At home, data center of CHINAUNICOM year power consumption 9,900,000,000 kilowatt hour, amounts to and need consume 920,000 tons of standard coals year; Data center of China Telecom year power consumption 11,200,000,000 kilowatt hour, amounts to and need consume 102.95 ten thousand tons of standard coals year.According to the portion report observation of Green Peace, to the year two thousand twenty, " energy consumption 2,000,000,000,000 kilowatt hours nearly of cloud computing (comprising data processing and communication network) exceed moral, method, add and the energy consumption summation of 4 states such as Brazil the main IT operator in the whole world.Therefore a kind of energy-conservation method is sought very necessary to the energy consumption reducing data center.

Various for cloud data center computational resource, storage resources and Internet resources etc., by Intel Virtualization Technology, are integrated together, form a resource pool by cloud computing.Because the kind of resource is many, scale large, therefore rational management resource, to the energy-conservation resource scheduling algorithm of structure, there is important researching value.Three aspects are mainly concentrated on to the research of data center energy-saving technology: chip-scale power-saving technology, architecture level power-saving technology and system-level power-saving technology.First two implements power-saving technology from hardware aspect, and the third dispatches to realize energy-conservation object to each resource from system level by the mode of software.The present invention, from system-level Energy Angle, studies the resource scheduling algorithm under cloud environment.

Summary of the invention

Technical problem to be solved by this invention is to overcome prior art deficiency, resource regulating method under a kind of cloud environment is provided, from system-level Energy Angle, under the prerequisite of the time of implementation and expense two QoS demand that meet cloud task, minimum for target with system total energy consumption, resource is dispatched, effectively can reduce the energy resource consumption of cloud data center.

The present invention specifically solves the problems of the technologies described above by the following technical solutions:

A resource regulating method under cloud environment, processed by the M obtained after a cutting task matching to N number of physical host, it is characterized in that, the concrete scheme of task matching obtains by solving following Mathematical Modeling:

Mnimize ${\mathrm{EC}}_{\mathrm{total}}=\underset{j=1}{\overset{N}{\mathrm{\Σ}}}\left({\mathrm{EC}}_j\right)$

Subject to

$\∀t_i,r_i,i=\mathrm{1,2},\·\·\·,M,j=\mathrm{1,2},\·\·\·,N,$

T _ijexec≤T _iexpt,cost _ijvm≤cost _iexpt，

In formula, EC _totalfor system total energy consumption, EC _jrepresent a jth physical host r _jenergy consumption, T _ijexec, T _iexptrepresent i-th task t respectively _iexpected time, expect the time of implementation, cost _ijvm, cost _iexptrepresent task t respectively _iexpectation execution cost, expect execution cost.

Above-mentioned Mathematical Modeling can adopt existing various algorithm to be optimized and solve, preferably, described Mathematical Modeling solve use genetic algorithm, the chromosome coding method of described genetic algorithm is as follows: adopt real coding mode, chromosomal length equals the quantity M of task, gene wherein and M task one_to_one corresponding; The physical host that the task of value corresponding to this gene of gene takies is numbered;

The fitness function of described genetic algorithm is as follows:

f=k _adapt/lg(EC _total+1)，

In formula, k _adaptfor adaptation parameter, EC _totalfor system total energy consumption.

Preferably, the selection operation in described genetic algorithm is concrete in accordance with the following methods: according to normal state selection opertor probable value, with roulette method choice two chromosomes; Normal state selection opertor computing formula as follows:

$P_s\left(X_k^t\right)=\frac{e^{-\frac{{[f\left(X_k^t\right)-\mathrm{\μ}]}^2}{2{\mathrm{\σ}}^2}}}{\underset{j=1}{\overset{\mathrm{popSize}}{\mathrm{\Σ}}}{e}^{-\frac{[{f\left(X_k^t\right)-\mathrm{\μ}]}^2}{2{\mathrm{\σ}}^2}}}$

Wherein, popSize is population scale, and μ represents the average fitness of t for population, σ ²represent the variance of each population at individual fitness, ${\mathrm{\σ}}^2=\frac{\underset{j=1}{\overset{\mathrm{popSize}}{\mathrm{\Σ}}}{[f\left(X_j^t\right)-\mathrm{\μ}]}^2}{\mathrm{popSize}}.$

Preferably, crossover and mutation operation in described genetic algorithm uses two-point crossover and even variation method, be specially: according to the probability of normal state crossover operator and the probability of normal mutation operator, crossover and mutation operation is carried out respectively to two chromosomes selected, generate two new chromosomes; Normal state crossover operator with normal mutation operator be expressed as follows respectively:

$P_c\left(X_k^t\right)=\left\{\begin{array}{cc}{k}_c(1-\sqrt{t/t_{\max }})e^{-\frac{{[f\left(X_k^t\right)-\mathrm{\&mu;}]}^2}{2{\mathrm{\&sigma;}}^2}},& f\left(X_k^t\right)\&GreaterEqual;\mathrm{\&mu;}\\ k_c,& f\left(X_k^t\right)<\mathrm{\&mu;}\end{array}\right.$

$P_m\left(X_k^t\right)=\left\{\begin{array}{cc}{k}_m(1-\sqrt{t/t_{\max }})e^{-\frac{{[f\left(X_k^t\right)-\mathrm{\&mu;}]}^2}{2{\mathrm{\&sigma;}}^2}},& f\left(X_k^t\right)\&GreaterEqual;\mathrm{\&mu;}\\ k_m,& f\left(X_k^t\right)<\mathrm{\&mu;}\end{array}\right.$

Wherein, k _cand k _mrepresent interaction coefficent and the coefficient of variation respectively, k _c, k _m∈ [0,1], t represents current iteration number of times, t _maxfor maximum iteration time.

The present invention, from system-level Energy Angle, under the prerequisite of the time of implementation and expense two QoS demand that meet cloud task, dispatches resource for target so that system total energy consumption is minimum, effectively can reduce the energy resource consumption of cloud data center; And adopt Revised genetic algorithum to carry out the search of optimum resource scheduling scheme further, the moderate individuality of fitness is improved to keep population diversity by the selection opertor introducing normal distribution, utilize normal distribution model according to fitness and evolutionary generation design crossover operator and mutation operator to keep population diversity and population structure, thus effectively improve convergence of algorithm performance, decrease the algorithm deadline.

Accompanying drawing explanation

Fig. 1 is the resource dispatching model under cloud environment;

The chromosome coding figure of the genetic algorithm used in Fig. 2 the inventive method;

Fig. 3 is the overall flow schematic diagram of the inventive method.

Embodiment

Below in conjunction with accompanying drawing, technical scheme of the present invention is described in detail:

As shown in Figure 1, adopt two-level scheduler model, concrete structure is as follows for resource dispatching model under cloud environment of the present invention:

First order scheduling realizes the mapping of user task to virtual machine, can be described as tlv triple, i.e. a S ₁={ T, V, f ₁.Wherein, T is user task set, and V is virtual machine set, f ₁for set of tasks is to a mapping of virtual machine set.With set T={t ₁, t ₂..., t _mrepresent the set that M task in cloud computing environment after Mapreduce cutting is formed.Each task t wherein _i(i ∈ [1, M]) can be with a quadruple notation: t _i=tid, tdata, tneed, tQoS}, wherein, tid represents mission number; Tdata represents the set of cloud task relevant information (or parameter), comprises the instruction strip number of task, the data volume of task read-write and the data volume of multiplexed transport; Tneed represents the primary demand set of task to resource, as CPU, internal memory, external memory, bandwidth etc.; TQoS represents the quality of service requirement set of finishing the work, as task execution time and expense.With set V={v ₁, v ₂..., v _mrepresent the set of the virtual machine corresponding with M task.Each virtual machine vm wherein _i(i ∈ [1, M]) can be with a quadruple notation: vm _i={ vmid, tdata, tneed, tQoS}, vmid represent the numbering of virtual machine, consistent with task of other parameter.Map f ₁: T → V represents that task arrives the mapping of virtual machine.Because first order scheduling of the present invention carrys out configuring virtual machine according to the demand of task, therefore set up the relation one to one of virtual machine and task.

Second level scheduling is that virtual machine is mapped to physical host layer, can be described below S by a tlv triple ₂={ V, R, f ₂.Wherein, V is virtual machine set, and R is physical host set, f ₂for virtual machine set is to mapping, i.e. an a kind of dispatching algorithm of physical host set.With set R={r ₁, r ₂..., r _nrepresent the set of the cloud computing resources that N number of physical machine forms.Each resource r wherein _j(j ∈ [1, N]) can be with a quadruple notation: r _j=rid, rtre, rcre, tdata}, wherein rid represents that physical host is numbered; Rtre represents and the hardware configuration that resource is total comprises CPU, internal memory, external memory, bandwidth etc.; Rcre represents the available configuration that resource is current; Rdata represents power consumption and the relevant information of resource, as the expense etc. calculating power consumption, store power consumption, transmit power consumption, disk read-write speed, various parts.Map f ₂: V → R represents the mapping of virtual machine to physical host.Second level scheduling is then the most suitable physical resource of goal seeking according to the configuration information of virtual machine with the energy-conservation of whole cloud environment.Primary study of the present invention realizes the method for second level scheduling, namely provides the mapping between virtual machine set to physical host set.For this reason, the present invention is minimum for target with system total energy consumption, sets up following resource dispatching model using the time of implementation and expense two QoS demand that meet cloud task as constraints:

Minimize ${\mathrm{EC}}_{\mathrm{total}}=\underset{j=1}{\overset{N}{\mathrm{\&Sigma;}}}\left({\mathrm{EC}}_j\right)$

Subject to

$\&ForAll;t_i,r_i,i=\mathrm{1,2},\&CenterDot;\&CenterDot;\&CenterDot;,M,j=\mathrm{1,2},\&CenterDot;\&CenterDot;\&CenterDot;,N,$

T _ijexec≤T _iexpt,cost _ijvm≤cost _iexpt，

In formula, EC _totalfor system total energy consumption, EC _jrepresent a jth physical host r _jenergy consumption, T _ijexec, T _iexptrepresent i-th task t respectively _iexpected time, expect the time of implementation, cost _ijvm, cost _iexptrepresent task t respectively _iexpectation execution cost, expect execution cost.Wherein, physical host r _jenergy consumption EC _jvarious existing method can be utilized to obtain, Calculation Method of Energy Consumption [the Beloglazov that the present invention preferably adopts the people such as Anton Beloglazov to propose, A., J.Abawajy andR.Buyya, Energy-aware resource allocation heuristics for efficient management of datacenters for cloud computing [J] .Future Generation Computer Systems, 2011,28:755 – 768].

Solve by being optimized above-mentioned Mathematical Modeling, such as adopt greedy algorithm, Lagrangian Arithmetic, Means of Penalty Function Methods, particle cluster algorithm etc., under the prerequisite of the time of implementation and expense two QoS demand that meet cloud task, the resource allocation proposal making system total energy consumption minimum can be obtained.The present invention preferably adopts genetic algorithm to carry out the search of optimum resource scheduling scheme.Genetic algorithm uses for reference a kind of global optimizing algorithm of living nature natural selection and evolution.The solution of problem is encoded into individuality (also known as chromosome) by it, multiple individuality forms initial population, based on the natural selection mechanism of the survival of the fittest, utilize the operators such as selection, intersection, variation that initial population is constantly evolved, direction towards optimal solution is moved, and finally searches the optimal solution of problem.

Use genetic algorithm to be optimized to solve, chromosome coding method must be determined and design suitable fitness function, chromosome coding method of the present invention is as shown in Figure 2: adopt real coding mode, chromosomal length equals the quantity M of task, gene wherein and M task one_to_one corresponding; The physical host (resource) that the task of value corresponding to this gene of gene takies is numbered.

Fitness function designed by the present invention is as follows:

f=k _adapt/lg(EC _total+1)，

In formula, k _adaptfor adaptation parameter, be used for ensureing that adaptive value can know display; EC _totalfor system total energy consumption.

According to above-mentioned chromosome coding method and fitness function, various existing genetic algorithm can be adopted to carry out the search of optimal solution.In order to improve convergence of algorithm performance, reduce the deadline, the present invention improves traditional genetic algorithm again further: improve the moderate individuality of fitness to keep population diversity by the selection opertor introducing normal distribution, utilizes normal distribution model to design crossover operator and mutation operator to keep population diversity and population structure according to fitness and evolutionary generation.Particularly, in the inventive method, select operation concrete in accordance with the following methods: according to normal state selection opertor probable value, with roulette method choice two chromosomes; Normal state selection opertor computing formula as follows:

$P_s\left(X_k^t\right)=\frac{e^{-\frac{{[f\left(X_k^t\right)-\mathrm{\&mu;}]}^2}{2{\mathrm{\&sigma;}}^2}}}{\underset{j=1}{\overset{\mathrm{popSize}}{\mathrm{\&Sigma;}}}{e}^{-\frac{[{f\left(X_k^t\right)-\mathrm{\&mu;}]}^2}{2{\mathrm{\&sigma;}}^2}}}$

Wherein, popSize is population scale, and μ represents the average fitness of t for population, σ ²represent the variance of each population at individual fitness, ${\mathrm{\&sigma;}}^2=\frac{\underset{j=1}{\overset{\mathrm{popSize}}{\mathrm{\&Sigma;}}}{[f\left(X_j^t\right)-\mathrm{\&mu;}]}^2}{\mathrm{popSize}}.$

Crossover and mutation operation uses two-point crossover and even variation method, is specially: carry out crossover and mutation operation according to the probability of normal state crossover operator and the probability of normal mutation operator respectively to two chromosomes selected, and generates two new chromosomes; Normal state crossover operator with normal mutation operator be expressed as follows respectively:

$P_c\left(X_k^t\right)=\left\{\begin{array}{cc}{k}_c(1-\sqrt{t/t_{\max }})e^{-\frac{{[f\left(X_k^t\right)-\mathrm{\&mu;}]}^2}{2{\mathrm{\&sigma;}}^2}},& f\left(X_k^t\right)\&GreaterEqual;\mathrm{\&mu;}\\ k_c,& f\left(X_k^t\right)<\mathrm{\&mu;}\end{array}\right.$

$P_m\left(X_k^t\right)=\left\{\begin{array}{cc}{k}_m(1-\sqrt{t/t_{\max }})e^{-\frac{{[f\left(X_k^t\right)-\mathrm{\&mu;}]}^2}{2{\mathrm{\&sigma;}}^2}},& f\left(X_k^t\right)\&GreaterEqual;\mathrm{\&mu;}\\ k_m,& f\left(X_k^t\right)<\mathrm{\&mu;}\end{array}\right.$

Wherein, k _cand k _mrepresent interaction coefficent and the coefficient of variation respectively, k _c, k _m∈ [0,1], t represents current iteration number of times, t _maxfor maximum iteration time.

Resource regulating method under cloud environment of the present invention, as shown in Figure 3, specifically comprises the following steps:

Step 1) chromosome coding: as shown in Figure 2, adopt real coding mode, chromosomal length equals the quantity M of task, gene wherein and M task one_to_one corresponding; The physical host (resource) that the task of value corresponding to this gene of gene takies is numbered;

Step 2) genetic algorithm initialization: initialization population scale is popSize, maximum iteration time t _max, make algorithm perform current algebraically t=0, population number m=0 of future generation;

Step 3) produces initial population P (t), i.e. the original allocation scheme of resource: by coding rule stochastic generation chromosome of step 1), judge whether the QoS demand meeting task, if met, is then added initial population; Until initial population scale reaches popSize;

Step 4) evaluates resource scheduling scheme: each chromosome is corresponding a kind of resource scheduling scheme, calculates each chromosomal fitness, body one by one the highest for fitness is directly copied to the next generation; Fitness function is expressed as follows:

f=k _adapt/lg(EC _total+1)

Wherein k _adaptfor adaptation parameter, be used for ensureing that adaptive value can know display, EC _totalfor system total energy consumption;

Step 5) is selected individual: according to normal state selection opertor from P (t) probable value, with roulette method choice two chromosomes; If the fitness of individuality is f (X _k), then individual X _kin t generation by the probability selected be:

$P_s\left(X_k^t\right)=\frac{e^{-\frac{{[f\left(X_k^t\right)-\mathrm{\&mu;}]}^2}{2{\mathrm{\&sigma;}}^2}}}{\underset{j=1}{\overset{\mathrm{popSize}}{\mathrm{\&Sigma;}}}{e}^{-\frac{[{f\left(X_k^t\right)-\mathrm{\&mu;}]}^2}{2{\mathrm{\&sigma;}}^2}}}$

In formula, popSize is population scale, and μ represents the average fitness of t for population, represent the variance of each population at individual fitness;

The individual crossover and mutation of step 6): adopt two-point crossover and even variation method, to two chromosomes selected in step 5) respectively according to normal state crossover operator probability and normal mutation operator probability to carry out crossover and mutation operation generation two new individual; Normal state crossover operator with normal mutation operator be expressed as follows:

$P_c\left(X_k^t\right)=\left\{\begin{array}{cc}{k}_c(1-\sqrt{t/t_{\max }})e^{-\frac{{[f\left(X_k^t\right)-\mathrm{\&mu;}]}^2}{2{\mathrm{\&sigma;}}^2}},& f\left(X_k^t\right)\&GreaterEqual;\mathrm{\&mu;}\\ k_c,& f\left(X_k^t\right)<\mathrm{\&mu;}\end{array}\right.$

$P_m\left(X_k^t\right)=\left\{\begin{array}{cc}{k}_m(1-\sqrt{t/t_{\max }})e^{-\frac{{[f\left(X_k^t\right)-\mathrm{\&mu;}]}^2}{2{\mathrm{\&sigma;}}^2}},& f\left(X_k^t\right)\&GreaterEqual;\mathrm{\&mu;}\\ k_m,& f\left(X_k^t\right)<\mathrm{\&mu;}\end{array}\right.$

Wherein, k _cand k _mrepresent interaction coefficent and the coefficient of variation respectively, k _c, k _m∈ [0,1], t represents current iteration number of times, t _maxfor maximum iteration time;

Step 7) individuality screening: judge whether two newly-generated chromosomes meet the QoS demand of task, if meet, compares father's individuality and the fitness of sub-individuality, high for fitness two individualities is added population P (t+1) of future generation; Do not meet, then turn to step 6);

Step 8) generates new population: judge population scale, if m<popSize, turn to step 5); Otherwise t=t+1, obtains resource scheduling scheme of new generation;

If step 9) t>t _max, termination of iterations, exports optimum individual in population, obtains the resource scheduling scheme making system total energy consumption minimum; Otherwise, go to step 4), carry out iterative operation.

The present invention, under the prerequisite of the time of implementation and expense two QoS demand that meet cloud task, adopts Revised genetic algorithum, minimum for target with system total energy consumption, resource is dispatched, can reduce the energy consumption of system while guaranteed qos demand, and algorithmic statement performance is good, required time is short.

Claims (2)

1. the resource regulating method under cloud environment, processed by the M obtained after a cutting task matching to N number of physical host, it is characterized in that, the concrete scheme of task matching obtains by solving following Mathematical Modeling:

Subject to

T _ijexec≤T _iexpt,cost _ijvm≤cost _iexpt，

In formula, EC _totalfor system total energy consumption, EC _jrepresent a jth physical host r _jenergy consumption, T _ijexec, T _iexptrepresent i-th task t respectively _iexpected time, expect the time of implementation, cost _ijvm, cost _iexptrepresent task t respectively _iexpectation execution cost, expect execution cost;

Described Mathematical Modeling solve use genetic algorithm, the chromosome coding method of described genetic algorithm is as follows: adopt real coding mode, chromosomal length equals the quantity M of task, gene wherein and M task one_to_one corresponding; The physical host that the task of value corresponding to this gene of gene takies is numbered;

The fitness function of described genetic algorithm is as follows:

f＝k _adapt/lg(EC _total+1)，

In formula, k _adaptfor adaptation parameter, EC _totalfor system total energy consumption;

Selection operation in described genetic algorithm is concrete in accordance with the following methods: according to normal state selection opertor probable value, with roulette method choice two chromosomes; Normal state selection opertor computing formula as follows:

Wherein, popSize is population scale, and μ represents the average fitness of t for population, σ ²represent the variance of each population at individual fitness, f (X _k) represent individual X in population _kfitness, represent individual X _kin t generation by the probability selected.

2. the resource regulating method as claimed in claim 1 under cloud environment, it is characterized in that, crossover and mutation operation in described genetic algorithm uses two-point crossover and even variation method, be specially: according to the probability of normal state crossover operator and the probability of normal mutation operator, crossover and mutation operation is carried out respectively to two chromosomes selected, generate two new chromosomes; Normal state crossover operator with normal mutation operator be expressed as follows respectively:

Wherein, k _cand k _mrepresent interaction coefficent and the coefficient of variation respectively, k _c, k _m∈ [0,1], t represents current iteration number of times, t _maxfor maximum iteration time.