CN109447264B - Virtual machine placement genetic optimization method based on VHAM-R model in cloud computing environment - Google Patents
Virtual machine placement genetic optimization method based on VHAM-R model in cloud computing environment Download PDFInfo
- Publication number
- CN109447264B CN109447264B CN201811079838.9A CN201811079838A CN109447264B CN 109447264 B CN109447264 B CN 109447264B CN 201811079838 A CN201811079838 A CN 201811079838A CN 109447264 B CN109447264 B CN 109447264B
- Authority
- CN
- China
- Prior art keywords
- placement
- host
- virtual machine
- availability
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 51
- 238000005457 optimization Methods 0.000 title claims abstract description 43
- 230000002068 genetic effect Effects 0.000 title claims abstract description 31
- 238000005265 energy consumption Methods 0.000 claims abstract description 58
- 230000008569 process Effects 0.000 claims abstract description 38
- 108090000623 proteins and genes Proteins 0.000 claims description 70
- 230000006870 function Effects 0.000 claims description 34
- 210000000349 chromosome Anatomy 0.000 claims description 24
- 238000004891 communication Methods 0.000 claims description 15
- 230000035772 mutation Effects 0.000 claims description 12
- 238000004364 calculation method Methods 0.000 claims description 9
- 230000013011 mating Effects 0.000 claims description 9
- 230000006872 improvement Effects 0.000 claims description 4
- 238000013507 mapping Methods 0.000 claims description 4
- 230000008901 benefit Effects 0.000 claims description 3
- 229960001948 caffeine Drugs 0.000 claims description 3
- 230000001934 delay Effects 0.000 claims description 3
- 238000003780 insertion Methods 0.000 claims description 3
- 230000037431 insertion Effects 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 3
- 230000001681 protective effect Effects 0.000 claims description 3
- 238000005215 recombination Methods 0.000 claims description 3
- 230000006798 recombination Effects 0.000 claims description 3
- 230000001172 regenerating effect Effects 0.000 claims description 3
- 230000008439 repair process Effects 0.000 claims description 3
- 230000003068 static effect Effects 0.000 claims description 3
- RYYVLZVUVIJVGH-UHFFFAOYSA-N trimethylxanthine Natural products CN1C(=O)N(C)C(=O)C2=C1N=CN2C RYYVLZVUVIJVGH-UHFFFAOYSA-N 0.000 claims description 3
- 230000003044 adaptive effect Effects 0.000 description 4
- 230000002759 chromosomal effect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000013468 resource allocation Methods 0.000 description 2
- 239000002699 waste material Substances 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000012634 fragment Substances 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/12—Computing arrangements based on biological models using genetic models
- G06N3/126—Evolutionary algorithms, e.g. genetic algorithms or genetic programming
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biophysics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Physics & Mathematics (AREA)
- Molecular Biology (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Physiology (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Genetics & Genomics (AREA)
- Multi Processors (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
A virtual machine placement genetic optimization method based on a VHAM-R model in a cloud computing environment comprises the following steps: the first step is as follows: the following formalized description is proposed for the virtual machine placement problem, the process is as follows: 1.1 defining a placement environment; 1.2 defining the resource state; 1.3 host availability; 1.4 calculating the electric energy consumption; 1.5 defining virtual machine placement; the second step is that: setting constraint conditions and optimization targets for virtual machine placement; the third step: creating a model: based on the constraint conditions and the optimization target of virtual machine placement given in the second step, establishing a virtual hierarchical structure model VHAM-R based on a Rendervous hash algorithm, and using the virtual hierarchical structure model VHAM-R to optimize and decide the selection process of the virtual machine to the host; the fourth step: and (3) improving the operation of the genetic algorithm based on the VHAM-R model. The invention improves the execution efficiency of the algorithm and the optimization of the finally obtained solution set.
Description
Technical Field
The invention relates to the field of virtual machine placement in a cloud computing environment, in particular to optimization and improvement of virtual machine placement through encoding, selection operation, cross operation and mutation operation of a genetic algorithm on a VHAM-R model.
Background
Cloud computing is derived from distributed computing and grid computing, is a computing mode completely based on the internet, and provides low-cost, high-reliability and scalable computing resources and services for users according to a pay-as-needed mode. The basic idea of cloud computing is to provide physical device support through huge data centers distributed around the world, and provide high-quality computing and storage services for users through the internet at a low price based on a virtualization technology. Virtual Machine Placement (Virtual Machine platform) is a binning problem between cloud data center Virtual machines and physical hosts, an important component of resource management and allocation in a cloud computing environment. The essence of the problem is that the virtual machine is placed on a better physical node through a reasonable distribution method, and meanwhile, the resource requirement and specific constraint condition for running the virtual machine need to be met, so that the problem is an NP-hard problem. The good virtual machine placement strategy can effectively improve the resource utilization rate of each physical host of the cloud data center, reduce the overall energy consumption of the cloud data center, ensure the usability of user requirements and the like.
Z Zhang, CC Hsu and the like take the scale of a data center, the workload of a host and the requirement change of computing resources into consideration, and an energy-saving framework for placing a virtual machine on the host is provided, so that the problems of minimum resource waste and minimum energy consumption of a resource allocation virtual machine are effectively solved. X Li and Z Qian et al investigated the problem of selecting the appropriate physical host to deploy a virtual machine at runtime. As the physical host resources have multidimensional property and the use imbalance of the multidimensional resources can cause resource waste, the multidimensional resource partitioning model is provided for balancing the utilization rate of the multidimensional resources and reducing the number of the running physical hosts, so that the energy consumption of the data center is reduced. The domestic Liqiang et al provides a model based on long-term load performance for the virtual machine placement problem in the cloud computing environment, and effectively reduces the number of used physical host nodes by combining a genetic algorithm of multi-objective optimization.
The Genetic Algorithm (GA) is proposed by Holland in 1975 by referring to natural selection of the biological evolution theory and the biological evolution process in genetics, and is an algorithm for finding the optimal solution of the NP-hard problem by simulating the biological evolution process. Compared with the common virtual machine Placement algorithm under cloud computing based on ANSYS cluster finite element analysis, the virtual machine Placement algorithm comprises the classic algorithms such as First In First Service (First In First Service), First adaptive algorithm (Fist discovery), First adaptive descent algorithm (Fist First discovery), Best adaptive algorithm (Best Fit), Best adaptive descent algorithm (Best Fit), Greedy Placement algorithm (Greedy Placement), and the like, and the genetic algorithm can better find the global optimal solution.
Disclosure of Invention
In order to place the group of virtual machine placement requests on the server nodes, the optimization target of high virtual machine placement availability with fewer working hosts and low energy consumption of the data center is realized on the premise of meeting the constraint conditions of placement, resources, communication accessibility and the like. The invention provides an improved genetic algorithm for processing the virtual machine placement problem, and the encoding mode, selection, mating process and variation mode of the virtual machine placement problem are improved based on a VHAM-R model, so that the execution efficiency of the algorithm is improved, and the finally obtained solution set is optimized.
The invention provides the following technical scheme for solving the technical problems:
a virtual machine placement genetic optimization method based on a VHAM-R model in a cloud computing environment comprises the following steps:
the first step is as follows: the following formalized description is proposed for the virtual machine placement problem, the process is as follows:
1.1 define the placement environment, and the data center has a physical host set PM ═ PM1,pm2,…,pmnN, the number of hosts is n, and a virtual machine set VM ═ VM needs to be placed1,vm2,…,vmmAnd where the number of virtual machines is m, and assuming that the number of virtual machines m is greater than or equal to the number of hosts n, defining a set of virtual machine placement groups P ═ { P ═ P1,p2,…,phH is the number of the placing groups;
1.2 defining resource states, vm for a given virtual machineiDefinition ofFor virtual machines vmiThe required resources of the CPU are used,for virtual machines vmiRequired memory resource, Vi-pesFor virtual machines vmiCPU utilization of Wi-ramFor virtual machines vmiMemory utilization of, for a given host pmjDefinition ofIs a main engine pmjThe current resources of the CPU are free from the CPU,is a main engine pmjFree resources of memory, Uj-pesIs a main engine pmjCPU utilization, Uj-ramIs a main engine pmjThe memory utilization rate of (1) then defines the host pmjResource utilization rate UjComprises the following steps:
Uj=αUj-pes+βUj-ram
0< α <1,0< β <1, and α + β ═ 1;
definition of TagijAt the current time t, the host pmjWhether or not to satisfy vm of virtual machineiResource requirements of, i.e.
1.3 host availability, availability of a node means the probability of a node working at any time during the entire service time, for any network component i, its availability AiObtained by the following formula:
wherein MTTF represents mean time to failure, MTTR represents mean time to repair, assuming that the value of server availability is known and the availability between servers is independent and independent of each other;
1.4 calculating the power consumption, and in a cloud data center with n running physical hosts, for any physical host pmjE PM, the power consumption at a certain time t is shown by the following formula:
wherein c isjFor static energy consumption marking, fj(t) is time t host pmjCPU frequency, CPU utilization of Uj-pes(t), k is a constant coefficient, i.e. the power consumption is to some extent based on a linear model of the CPU utilization,
1.5 defining virtual machine Placement, VM set by Placement group pkE.g. P, selecting a host in the corresponding physical host set to complete placement mapping, and defining a virtual machine placement matrix M by meeting various constraint conditions in the placement process as much as possiblek[i][j]If M is presentk[i][j]1 denotes the placement group pkPlace virtual machine j on physical host i, otherwise, if M isk[i][j]0 denotes the placement group pkIn (1), virtual machine j is not placed on physical host i,
the second step is that: setting constraint conditions and optimization targets for virtual machine placement, wherein the process is as follows:
2.1 regarding the virtual machine placement problem in the cloud environment, not only needs to consider meeting the requirements of virtual machine resources, but also needs to consider how to reduce the energy consumption of a data center and efficiently utilize the resources, and in addition, needs to consider the availability problem of the placement request; therefore, the constraints to be considered are: the maximum use number of the server nodes is minimum, the energy consumption is minimum, the load is balanced, and the availability of the placement request is high, and the following constraint conditions are provided:
2.1.1 placing constraints, arbitrary virtual machine vmiUnder the same placing group, the two can be placed in one and only oneA server node;
the constraints represent:
In the same placement group, a single virtual machine is considered to be deployed and operated on only one server node;
2.1.2 resource constraints, for any server node, the consumption of each resource type should not exceed the upper limit, defining server pmjRespectively, of CPU and memory capacityAndrepresents;
the constraints represent:
The parameter is a constant coefficient, a part of resources are required to be reserved by the server node to ensure the normal operation of the server node, r is less than or equal to 1, and the normal value is 0.8;
2.1.3 reachability constraints, defining a function F (m, n, D) for representing reachability of inter-node communications, for any link (m, n) e.L, if communication delays of points m and n are at most D, the function F (m, n, D) returns 1, otherwise returns 0;
2.2, the virtual machine placement problem has a plurality of optimization targets, and the virtual machine placement problem is optimized and researched in two aspects of availability and energy consumption;
2.2.1 usability optimization
Assuming that the user request consists of a virtual machine between n different VM pairs with associated communication requirements, it is placed on the same server node pmjNot more than once, the usability of the placement cannot be improved because when pmjAt failure, all placement is at pmjWill fail simultaneously, and thus it is desirable to try to get vm toiPlaced on different nodes to increase availability; by HiTo represent placement of virtual machines vmiThe maximum number of nodes, i.e. HiRepresents vmiThe maximum number of server nodes that can be placed; definition ofThe node number of the n virtual machines is H at most;
the availability definition and calculation of virtual machine placement can be divided into three categories: single placement, fully protected placement, and partially protected placement;
2.2.1.1 Single Placement
Single placement means that each virtual machine is placed on only one server node, i.e. H ═ 1, in the case of single placement, if the availability of n server nodes is a respectively1,A2,…,AnK virtual machines (n ≦ k) are placed on the n nodes, then the availability of the virtual machine placement scheme may be used as ApExpressed, defined as follows:
since the request contains k virtual machines, the probability that all the k virtual machines are running needs to be considered when calculating the availability;
2.2.1.2 complete protective Placement
Full protection placement refers to placement of arbitrary virtual machinesAre all placed in group pi(i is more than or equal to 1 and less than or equal to H) are placed on H different nodes; thus, a complete protection can be consideredThe placement plan P can be composed of H single placement plans, and in each single placement plan, the virtual machine pair should satisfy placement, resource and communication accessibility constraints;
availability of a full protection placement solution is the probability that there is at least one placement group working within the life cycle of the service, and the availability is calculated as follows:
2.2.1.3 partial protection Placement
Partial protection placement means presence of virtual machine vmiE.g. VM, placed on less than H different nodes, i.e. two or more placement groups will virtual machine VMiPlaced on the same node and having a certain virtual machine vmjE.g. VM, so that H is greater than 1, and under the condition of placement of partial protection, if a virtual machine is placed on less than H nodes, the virtual machine can be considered to be placed by a plurality of placement groups together; the availability of the server node cannot be directly calculated by the formula in 2.2.1.2 because the availability of the server node where the shared virtual machine is placed is calculated twice; redefining the operator to handle the placement; suppose there are n nodes pm1,pm2,…,pmnTheir availability is A1,A2,…,An(ii) a For availability of AxNode pmxThe following definitions are given for the operators:
then according to the formula in the above, defineFor operations between different sets, the availability of protection placement is calculated by the following formula:
2.2.2 energy consumption optimization
According to the formula in 1.4, during the period T, the physical host pmjTotal energy consumption ofExpressed as:
therefore, the following formula shows that the total energy consumption E of the servers of the data center is within the T periodTFor the sum of the energy consumptions of the various running servers:
the third step: creating a model
Based on the constraint conditions and the optimization target of virtual machine placement given in the second step, a virtual hierarchical structure model (VHAM-R) based on a Rendervous hash algorithm is established for optimizing and deciding the selection process of the virtual machine to the host, and the steps are as follows:
3.1 initializing a host set PM and a virtual machine set VM, wherein the number of the virtual machines to be placed in the group H at most and the availability set A of host nodes are obtained, if the number n of the host is less than 4, the step 3.2 is carried out, otherwise, the step 3.3 is carried out;
3.2 when the number n of hosts is less than 4, i.e. the number 2 x 2 of the minimum bi-level virtual structure cannot be constructed, defining that the host has a set W of assigned weight scores corresponding to each hosti={wi1,wi2,…,wikWhere k.gtoreq.n defines wijFor virtual machines vmiAt host pmjA weight score of (1), wij=h(vmi,pmj) Wherein the function h (vm, pm) is a hash function agreed in the Renderkvous hash algorithm, and then passes through h (vmi,pmj) Virtual machine vmiIs assigned to a weight wijMaximum host pmjIf the host pmkIs h times that of other hosts, pmkThe equal share is divided into h shares, and the probability of the virtual machine distributed to the host is h times of that of other hosts;
3.3 the number n of the hosts is more than or equal to 4, for the host cluster division, firstly, a constant z is selected, namely the number of the hosts in each cluster is z, and the host set is C-ceiling (n/z), wherein the ceiling function represents that the value of dividing n by z is rounded up to the nearest integer, C0={cpm1,cpm2,…,cpmz},C1={cpmz+1,cpmz+2,…, cpm2z}, … until each host belongs to a cluster, each cluster being the lowest node in the virtual hierarchy;
the fourth step: the genetic algorithm operation improvement based on the VHAM-R model comprises the following steps:
4.1 for solving the process of placing the virtual machine in the server node, a grouping coding mode based on the host node cluster is provided, P represents an individual for the placing group, CiRepresenting that the host clusters correspond to chromosomes and the host on each host cluster corresponds to a gene;
4.2, selecting individuals containing good genes in the population through a fitness function set by the selection operation, and taking the energy consumption of the data center and the placement availability of the virtual machines as optimization targets;
4.3 crossover operation in genetic algorithm is to simulate the mating process between individuals in the biological world, and achieve gene recombination between parents through mating, so that the offspring obtains a new chromosome containing an excellent gene and generates more excellent offspring;
4.4 mutation operations in genetic algorithms.
Further, in step 3.3, the process of dividing the host cluster includes:
3.3.1 determining virtual leaf node sectors and the depth of a virtual hierarchical structure, selecting the leaf number f of each sub-node sector in the virtual hierarchical structure, wherein f is an integer, and selecting proper f and z can enable the obtained algorithm benefit, load balance degree and the like to be approximate to expectations; according to the leaf number f of the node sector and the number z of the host clusters, the depth d of the virtual hierarchical structure can be obtained:
fd≥c
wherein d is the smallest positive integer, and d is calculated so that the above formula holds;
3.3.2 numbering each virtual leaf node sector, numbering each sector uniformly by adopting natural numbers respectively, from 0, 1, 2, …, f-1;
3.3.3 for a certain virtual machine vmiFor any virtual node s, there is a corresponding weight wis=h(vmi,s),h(vmiS) includes a hash function of a convention, such as hash32, hash64, etc., that passes through h (vm) at each level of the virtual hierarchyiS) calculating the weight of each virtual node, selecting the node with the highest score to continue layering downwards until the real host node cluster Cx to the bottom layer is selected;
3.3.4 virtual machine vmiSelecting a real host node cluster CxLater, when real node selection is carried out, the fact that any real node cluster C exists is assumedxHost node cpm of (1)xz+jAll have a corresponding weight score Wi(xz+j)=H(vmi,cpmxz+j)*Tagi(xz+j)If Tag is false, the value is 0, and if true, the value is 1; wherein the virtual machine vm is usediCpm assigned to hostxz+jThen, H (vm)i,cpmxz+j) For the same period of T, EoldAnd assigning virtual machines vmiReal host node cluster CxRatio of total power consumption of to the host pmxz+jResource utilization rate Uxz+jProduct of the difference with 1 and the corresponding weight constant and sum of host availability and coefficient product:
wherein Exz+jFor the cpm of the host in the T time periodxz+jEnergy consumption of EoldMeans that in the same T time period, when no new virtual machine is allocated, the virtual machine is realHost node cluster CxEnergy consumption of (2); α, β, γ are weights representing the three; a. thexz+jCpm for the hostxz+jAvailability of (2);
3.3.5 loop through steps 3.3.3-3.3.4 until all virtual machines vmiFinally, the weighting score W is selectedi(xz+j)The highest host node completes the allocation.
Still further, in step 4.2, the process of selecting operation is as follows:
4.2.1 when H1, for each individual, there is a virtual machine placement case, so its availability is calculated as a single placement, i.e. for individual P, its placement availabilityWherein the individual comprises n genes;
4.2.2 when H >1, availability for that H individual needs to be handled in a computationally intensive manner for protected placement;
4.2.2.1 for arbitrary virtual machines vmiE.g. VM, each of which is placed a group piThe virtual machines are placed on H different nodes, i is more than or equal to 1 and less than or equal to H, namely for H individuals, all the virtual machines on each gene are different, and under the condition, the usability of the H placement groups is processed according to a calculation mode of complete protection placement;
4.2.2.2 virtual machine vm if presentiE VM, virtual machine VM in two or more individualsiPlaced on the same gene, in which case the availability of the H placement groups is handled in a computational manner of partial protection placement;
4.2.3 combine data center energy consumption optimization to give the fitness function of the following genetic algorithm:
wherein E isminThe minimum value of the energy consumption of the data center in the T time period is obtained;energy consumption for a single individual; when H ═ 1, availability is calculated in a single placement manner; when H is present>1, the availability calculation is divided into full protection placement and partial protection placement, wherein x is an individual group with the number of H; for an individual or an individual group x, if the lower the energy consumption of the data center and the higher the availability, the larger the value of the fitness function is, and the higher the selected probability that the good genes are inherited to the next generation is.
Furthermore, in step 4.3, the process of the interleaving operation is as follows:
4.3.1 first choose two parents to be mated, named X, Y, randomly choose a certain node cluster containing one or more genes in the X parent as a part to be crossed, insert the node cluster, namely all the genes in the node cluster, into the cross point of the Y parent, and then generate new filial generation containing X, Y parent genes;
4.3.2 after completing gene insertion, because the invention uses the chromosome grouping coding mode based on the host cluster, the same host cluster may appear, if the situation appears, the inserted gene is merged into the original host cluster;
4.3.3 if the same two virtual machines exist on different host nodes, temporarily removing the host nodes which previously contain the same virtual machines from the chromosome codes;
4.3.4, the host nodes are temporarily eliminated, and virtual machine nodes which are not deployed by other hosts are possibly included, for the case, the virtual machines need to be recoded into the host nodes, and the genes which meet the constraint conditions and have the lowest energy consumption and the highest availability in the chromosome are preferably selected to complete the distribution;
4.3.5 if all genes do not meet the requirements, regenerating a new gene segment according to the VHAM-R model, interchanging two parent individuals, and repeating the crossing process to generate a second filial generation individual.
In the step 4.4, the mutation process includes:
4.4.1 determining the individual chromosome gene to be mutated by the mutation function, as shown in the following formula:
wherein U isj-pes、Uj-ramRespectively the CPU and the internal memory utilization ratio of the host,is a set parameter;
4.4.2 selection of fc(j) The smaller gene is deleted, so that the gene with lower utilization rate is deleted every time,
4.4.3 inserting the virtual machine on the gene into other genes by a 4.3 cross operation method.
The invention has the beneficial effects that: and improving the encoding mode, selection, mating process and variation mode of the VHAM-R model based on the VHAM-R model, thereby improving the execution efficiency of the algorithm and the optimization of the finally obtained solution set.
Drawings
FIG. 1 is a VHAM-R model of the invention.
FIG. 2 shows the host cluster-based chromosomal coding of the present invention.
FIG. 3 is a schematic diagram of the crossover operation of the improved genetic algorithm of the present invention.
FIG. 4 is a schematic diagram of the variant operation of the improved genetic algorithm of the present invention.
Detailed Description
The invention is further described below with reference to the accompanying drawings.
Referring to fig. 1 to 4, a virtual machine placement genetic optimization method based on a VHAM-R model in a cloud computing environment includes the following steps:
the first step is as follows: the following formalized description is proposed for the virtual machine placement problem, the process is as follows:
1.1 define the placement environment, and the data center has a physical host set PM ═ PM1,pm2,…,pmnN, the number of hosts is n, and a virtual machine set VM ═ VM needs to be placed1,vm2,…,vmmAnd where the number of virtual machines is m, and assuming that the number of virtual machines m is greater than or equal to the number of hosts n, defining a set of virtual machine placement groups P ═ { P ═ P1,p2,…,phH is the number of placed groups.
1.2 defining resource states, vm for a given virtual machineiDefinition ofFor virtual machines vmiThe required resources of the CPU are used,for virtual machines vmiRequired memory resource, Vi-pesFor virtual machines vmiCPU utilization of Wi-ramFor virtual machines vmiThe memory utilization ratio. For a given host pmjDefinition ofIs a main engine pmjThe current resources of the CPU are free from the CPU,is a main engine pmjFree resources of memory, Uj-pesIs a main engine pmjCPU utilization, Uj-ramIs a main engine pmjThe memory utilization rate of (1) then defines the host pmjResource utilization rate UjComprises the following steps:
Uj=αUj-pes+βUj-ram
0< α <1,0< β <1, and α + β ═ 1.
Definition of TagijAt the current time t, the host pmjWhether or not to satisfy vm of virtual machineiResource requirements of, i.e.
1.3 host availability, availability of a node means the probability of a node working at any time during the whole service time, for any networkNetwork component i, availability A thereofiCan be calculated by the following formula:
wherein MTTF represents mean time to failure, MTTR represents mean time to repair, assuming that the value of server availability is known and the availability between servers is independent and independent of each other;
1.4 calculating the power consumption, and in a cloud data center with n running physical hosts, for any physical host pmjE PM, the power consumption at a certain time t is shown by the following formula:
wherein c isjFor static energy consumption marking, fj(t) is time t host pmjCPU frequency, CPU utilization of Uj-pes(t), k is a constant coefficient, namely the power consumption is a linear model based on the CPU utilization rate to a certain extent;
1.5 defining virtual machine Placement, VM set by Placement group pkE, selecting a host in a corresponding physical host set to complete placement mapping, wherein the host belongs to P, and various constraint conditions in the placement process need to be met as much as possible; defining a virtual machine placement matrix Mk[i][j]If M is presentk[i][j]1 denotes the placement group pkPlace virtual machine j on physical host i, otherwise, if M isk[i][j]0 denotes the placement group pkIn (1), virtual machine j is not placed on physical host i.
The second step is that: setting constraint conditions and optimization targets for virtual machine placement, wherein the process is as follows:
2.1 regarding the virtual machine placement problem in the cloud environment, not only needs to consider meeting the requirements of virtual machine resources, but also needs to consider how to reduce the energy consumption of a data center and efficiently utilize the resources, and in addition, needs to consider the availability problem of the placement request; therefore, constraints that need to be considered are as long as: the maximum use number of the server nodes is minimum, the energy consumption is minimum, the load is balanced, and the availability of placing the request is high; for the above points, the following constraints are proposed:
2.1.1 placing constraints, arbitrary virtual machine vmiUnder the same placement group, it can be and can only be placed on one server node.
The constraints represent:
Description of the drawings: generally, in the same placement group, a single virtual machine is considered to be deployed and operated on only one server node;
2.1.2 resource constraint, for any server node, the consumption of each resource type should not exceed the upper limit, and the server resources are generally divided into CPU, memory, network bandwidth, disk resources, etc.; considering the resource condition of CPU and memory, defining server pmjRespectively, of CPU and memory capacityAndand (4) showing.
The constraints represent:
Description of the drawings: wherein, the parameter r (less than or equal to 1) is a constant coefficient, a part of resources need to be reserved by the server node to ensure the normal operation of the server node, and the normal value is 0.8;
2.1.3 reachability constraints, defining a function F (m, n, D) for representing reachability of inter-node communications, for any link (m, n) e.L, if communication delays of points m and n are at most D, the function F (m, n, D) returns 1, otherwise returns 0;
2.2 the optimization targets of the virtual machine placement problem are numerous, and typically comprise energy consumption optimization, network flow optimization, resource allocation optimization, availability optimization, performance optimization and the like; selecting two aspects of availability and energy consumption to carry out optimization research on the virtual machine placement problem;
2.2.1 usability optimization
Suppose a user request consists of a virtual machine between n different VM pairs with associated communication requirements (considering communication reachability), which is placed on the same server node (say pm)j) Not more than once, the usability of the placement cannot be improved because when pmjAt failure, all placement is at pmjThe virtual machines on will fail at the same time; therefore, it is necessary to try to make vm as small as possibleiPlaced on different nodes to increase availability. By HiTo represent placement of virtual machines vmiThe maximum number of nodes, i.e. HiRepresents vmiMaximum number of server nodes that can be placed, defineThe node number of the n virtual machines is H at most;
the availability definition and calculation of virtual machine placement can be divided into three categories: single placement, fully protected placement, partially protected placement;
2.2.1.1 Single Placement
Single placement means that each virtual machine is placed on only one server node, i.e. H ═ 1, in the case of single placement, if the availability of n server nodes is a respectively1,A2,…,AnK virtual machines (n ≦ k) are placed on the n nodes, then the availability of the virtual machine placement scheme may be used as ApExpressed, defined as follows:
since the request contains k virtual machines, the probability that all the k virtual machines are running needs to be considered when calculating the availability;
2.2.1.2 complete protective Placement
Full protection placement refers to placement of arbitrary virtual machinesAre all placed in group pi(1. ltoreq. i.ltoreq.H) are placed on H different nodes. Therefore, it can be considered that a full protection placement solution P may be composed of H single placement solutions, and within each single placement solution, placement, resource, and communication reachability constraints should be satisfied between pairs of virtual machines;
availability of a full protection placement solution is the probability that there is at least one placement group working within the life cycle of the service, and the availability is calculated as follows:
2.2.1.3 partial protection Placement
Partial protection placement means presence of virtual machine vmiE.g. VM, placed on less than H different nodes (i.e. two or more placement groups will virtual machine VM)iPlaced on the same node) and there is a certain virtual machine vmjBelongs to VM, so that H is more than 1; under the condition of partial protection placement, if one virtual machine is placed on less than H nodes, the virtual machine can be considered to be placed together by a plurality of placement groups; its availability cannot be calculated directly by the formula in 2.2.1.2 because the availability of the server node where the shared virtual machine is placed would be calculated twice. To handle this type of placement, operators are redefined. Suppose there are n nodes pm1,pm2,…,pmnTheir availability is A1,A2,…,An(ii) a For availability of AxNode pmxThe following definitions are given for the operators:
then according to the above formula, defineFor operations between different sets, the availability of partial protection placement can be calculated by the following formula:
2.2.2 energy consumption optimization
In a cloud computing environment, energy consumption generated by a data center is mainly energy consumption of various devices, including a server, a storage device, a network communication device and the like; the server accounts for the vast majority of energy consumption, the virtual machine placement is optimized from the energy consumption perspective, and a certain optimization target can be achieved by directly or indirectly reducing the starting operation number of the server mainly by reducing the energy consumption of server equipment;
according to the formula in 1.4, during the period T, the physical host pmjTotal energy consumption ofExpressed as:
therefore, the following formula shows that the total server energy consumption E of the data center is obtained in the T periodTThe sum of the energy consumption of each running server;
the third step: creating a model
Based on the constraint conditions and the optimization target of virtual machine placement given in the second step, a virtual hierarchical structure model (VHAM-R) based on a Rendervous hash algorithm is established for optimizing and deciding the selection process of the virtual machine to the host, and the basic idea of the Rendervous hash algorithm is that for each site SjAnd each object thereof OiThe corresponding weight can be calculated by the appointed hash function, and for each site SjSelecting the object O with the greatest weightmAnd is combined with OmTo station Sj;
3.1 initializing a host set PM, a virtual machine set VM, the maximum number of groups H placed by the virtual machines and an availability set A of host nodes, and entering a step 3.2 if the number n of the host machines is less than 4, or entering a step 3.3 if the number n of the host machines is not more than 4. Referring to fig. 1, the number of hosts n in the figure is 108, and step 3.3 is entered;
3.2 when the number n of hosts is less than 4, i.e. the number 2 x 2 of the minimum bi-level virtual structure cannot be constructed, defining that the host has a set W of assigned weight scores corresponding to each hosti={wi1,wi2,…,wik(where k.gtoreq.n). Definition of wijFor virtual machines vmiAt host pmjA weight score of (1), wij=h(vmi,pmj). The function h (vm, pm) includes a predetermined hash function, such as hash32, hash64, etc. . Wherein the function h (vm, pm) is a hash function appointed in the Renderzwaus hash algorithm, and then passes through h (vm)i,pmj) Function-to-virtual machine vmiIs assigned to a weight wijMaximum host pmj. If the host pmkIs h times that of other hosts, pmkIs divided equally into h portions, e.g. pmk1,pmk2,…,pmkh. Obviously, the probability of the virtual machine being allocated to the host is h times that of other hosts;
3.3 the number n of the host computers is more than or equal to 4, and the cluster groups of the host computers are divided. First, a constant z is selected, i.e. the number of hosts in each cluster is z, which is known as 4 in fig. 1. The host set is changed to ceil (108/4) to 27 according to c (ceil (n/z) ═ ceil (108/4) (where the ceil function indicates that the value of n divided by z is upRounded to the nearest integer). C0={cpm1, cpm2,…,cpmz},C1={cpmz+1,cpmz+2,…,cpm2zAnd … until each host belongs to a cluster. Each cluster is a bottommost node in the virtual hierarchical structure;
3.3.1 virtual leaf node sectors and virtual hierarchy depth determination. The leaf number f of each child node sector in the virtual hierarchical structure is selected, wherein f is a one-bit integer generally, and the selection of proper f and z can make the obtained algorithm benefit, load balance degree and the like closer to the expectation. As shown in fig. 1, the leaf number of each child node sector is 3, and according to the leaf number of the node sector 3 and the number of host clusters 27, the depth h of the virtual hierarchical structure can be obtained:
fd≥c
wherein d is the minimum positive integer, and d is solved to be 3;
3.3.2 each virtual leaf node sector number. Generally, each sector is numbered uniformly by adopting natural numbers, namely 0, 1, 2, … and f-1;
3.3.3 for a certain virtual machine vmiFor any virtual node s, there is a corresponding weight wis=h(vmi,s),h(vmiS) includes a hash function of a convention, such as hash32, hash64, etc., which may be h (vm) at each level of leaf sectors of the virtual hierarchyiS) calculate the virtual node weights, as in FIG. 1, starting from the root node because h (vm)i,0)>max{h(vmi,1),h(vmi2), thus selecting (0)1The node continues to go down; in the second level of three child nodes, because of h (vm)i,00)>max{h(vmi,01),h(vmi02) } thus select (00)1The node continues to go down; in the third layer of three child nodes, because h (vm)i,000)>max{h(vmi,001),h(vmi002) }, thus selecting (000)1Nodes, i.e. C0The true position is a node cluster, and then the next step of selection is carried out;
3.3.4 virtual machine vmiSelecting a real host node cluster C0Then, inWhen selecting the real nodes, the method assumes that the real nodes are randomly in the real node cluster C0Host node cpm of (1)j(1. ltoreq. j. ltoreq.z), all have a corresponding weight score Wij=H(vmi,cpmj)*Tagij(if Tag is false, it is 0, and if true, it is 1). Wherein the virtual machine vm is usediCpm assigned to hostjThen, H (vm)i,cpmj) For the same period of T, EoldAnd assigning virtual machines vmiReal host node cluster C0Ratio of total power consumption of to the host pmjResource utilization rate UjProduct of the difference with 1 and the corresponding weight constant and sum of host availability and coefficient product:
wherein EjFor the cpm of the host in the T time periodjEnergy consumption of EoldMeans that when no new virtual machine is allocated in the same T time period, the real host node cluster C0Energy consumption of (2). Alpha, beta, and gamma are weights representing the three, AjIs a main unit pmjHost availability. The 2 nodes in fig. 1 are the nodes selected finally;
3.3.5 loop through steps 3.3.3-3.3.4 until all virtual machines vmiFinally, the weighting score W is selectedi(xz+j)The highest host node completes the distribution;
the fourth step: genetic algorithm operation improvement based on VHAM-R model
An improved genetic algorithm is provided for processing the virtual machine placement problem, and the encoding mode, the selection, the mating process and the variation mode of the virtual machine placement problem are improved based on a VHAM-R model, so that the execution efficiency of the algorithm and the optimization of a finally obtained solution set are improved;
4.1 the coding mode of the chromosome has important influence on the search effect and the algorithm efficiency of the genetic algorithm, the realization process of the coding is the process of mapping the problem solution to the chromosome, and the invention provides the process of solving the virtual machine and placing the virtual machine to the server node based on the hostAnd (4) a grouping coding mode of the node cluster. P represents a placing group, CiIt means that the host clusters correspond to chromosomes, and the host on each host cluster corresponds to genes. FIG. 2 is a specific example of chromosomal encoding;
the data center in fig. 2 contains two placement groups p1 and p2, the placement request contains 8 virtual machines and a plurality of physical hosts, each placement group is composed of a plurality of host node clusters, each host node cluster has different numbers of host nodes, and each host node has different numbers of virtual machines. As in the left half of the figure, containing host clusters c1, c2, c4, host node E in host cluster c1, host node F in host cluster c2, and host node D in host cluster c 4. Virtual machines 2, 5, 4 are placed in host node E, virtual machines 3, 6, 7 are placed in host node F, and virtual machines 1, 8 are placed in host node D, the placement set corresponding to 3 chromosome basis factors, i.e., chromosome EFD. As in the right half of the figure, containing host clusters C0, C3, C5, host node A, B in host cluster C0, host node G in host cluster C3, and host node C in host cluster C5. Virtual machines 7, 6 are placed in host node a, virtual machines 2, 4 are placed in host node B, virtual machines 1, 5 are placed in host node G, and virtual machines 3, 8 are placed in host node C, the placement group corresponding to a chromosome base factor of 4, i.e., chromosome ABGC. Changing an operation object from a single virtual machine to a host cluster containing a virtual machine group by using a chromosome grouping coding mode based on the host cluster;
4.2 the selection operation of the genetic algorithm is to simulate the excellence and the disadvantage of organisms in the nature, and the selection operation can select individuals with higher adaptability and ensure that better genes can be smoothly inherited to the next generation;
selecting individuals containing excellent genes in the population through a fitness function set by the selection operation, and taking the energy consumption of a data center and the placement availability of a virtual machine as optimization targets;
4.2.1 when H1, for each individual, there is a virtual machine placement case, so its availability is calculated as a single placement, i.e. for individual p, its placement availabilityWherein the individual comprises n genes (host nodes);
4.2.2 when H >1, the availability for that H individuals (placement group) needs to be handled in a way that protected placement is calculated;
4.2.2.1 for arbitrary virtual machines vmiE.g. VM, each of which is placed a group pi(i is more than or equal to 1 and less than or equal to H) are placed on H different nodes, namely for H individuals, all virtual machines on each gene are different, and in this case, the availability of the H placement groups is processed according to a calculation mode of complete protection placement;
4.2.2.2 virtual machine vm if presentiE.g. VM, two or more individuals (placement group), virtual machine VMiPlaced on the same gene, in which case the availability of the H placement groups is handled in a computational manner of partial protection placement;
4.2.3 combine data center energy consumption optimization to give the fitness function of the following genetic algorithm:
wherein E isminThe minimum value of the energy consumption of the data center in the T time period is obtained;energy consumption for a single individual; when H ═ 1, availability is calculated in a single placement manner; when H is present>1, the availability calculations are divided into full protection placement and partial protection placement, where x is the number H of individual clusters. For an individual or an individual group x, if the energy consumption of the data center is lower and the availability is higher, the value of the fitness function is larger, and the selected probability of the good genes being inherited to the next generation is higher;
4.3 crossover operation in genetic algorithm is to simulate the process of mating between individuals in the biological world, and achieve recombination of genes between parents through mating, so that offspring can obtain new chromosomes containing excellent genes to generate more excellent offspring, which is specifically described with reference to fig. 3;
4.3.1 first pick two parents to be mated, named X, Y in fig. 3, randomly pick a certain node cluster containing one or more genes in the X parent as a part to be crossed, insert the node cluster, i.e. all genes in it, into the Y parent cross point, and at this time, generate a new offspring containing X, Y parent genes;
4.3.2 after completing gene insertion, because the invention uses the chromosome grouping coding mode based on the host cluster, the same host cluster may appear, if the situation appears, the inserted gene is merged into the original host cluster;
4.3.3 if the same two virtual machines exist on different host nodes, temporarily removing the host nodes which previously contain the same virtual machines from the chromosome codes, wherein virtual machines with repeated numbers appear on B, E and G in the figure 3, and removing the host machines B and G;
4.3.4, are temporarily dropped from the host node, may contain virtual machine nodes that are not deployed by other hosts,
in FIG. 3, the No. 1 virtual needs to be recoded into the host node, and the gene A is selected according to the constraint condition to complete the distribution;
4.3.5 if all genes do not meet the requirements, regenerating a new gene fragment according to the VHAM-R model. Interchanging two parent individuals, and repeating the intersection process to generate a second child individual;
4.4 mutation operation in the genetic algorithm is also obtained by simulating the evolution process of nature, the mutation is an important link in the genetic algorithm, and genes (including high-quality genes and poor-quality genes) which never appear in the parent individuals can be obtained through mutation with small probability, and the detailed description of the mutation operation is combined with the figure 4;
4.4.1 determining the individual chromosome gene to be mutated by the mutation function, as shown in the following formula.
Wherein U isj-pes、Uj-ramRespectively the CPU and the memory utilization rate of the host;
4.4.2 selection of fc(j) Deleting smaller genes to ensure that each time of deletion is poorer genes with lower utilization rate, wherein the variation function of the gene C in the figure 4 is smaller, and the gene C is removed;
4.4.3 the virtual machine No. 3 and the virtual machine No. 8 on the gene C in the figure 4 are coded again by a 4.3 cross operation method, the virtual machine No. 3 is inserted into the gene A, and the virtual machine No. 8 is inserted into the gene G.
Claims (2)
1. A virtual machine placement genetic optimization method based on a VHAM-R model in a cloud computing environment is characterized by comprising the following steps:
the first step is as follows: the following formalized description is proposed for the virtual machine placement problem, the process is as follows:
1.1 define the placement environment, and the data center has a physical host set PM ═ PM1,pm2,...,pmnN, the number of hosts is n, and a virtual machine set VM ═ VM needs to be placed1,vm2,...,vmmAnd where the number of virtual machines is m, and assuming that the number of virtual machines m is greater than or equal to the number of hosts n, defining a set of virtual machine placement groups P ═ { P ═ P1,p2,...,phH is the number of the placing groups;
1.2 defining resource states, vm for a given virtual machineiDefinition ofFor virtual machines vmiThe required resources of the CPU are used,for virtual machines vmiRequired memory resource, Vi-pesFor virtual machines vmiCPU utilization of Wi-ramFor virtual machines vmiMemory utilization of, for a given host pmjDefinition ofIs a main engine pmjThe current resources of the CPU are free from the CPU,is a main engine pmjFree resources of memory, Uj-pesIs a main engine pmjCPU utilization, Uj-ramIs a main engine pmjThe memory utilization rate of (1) then defines the host pmjResource utilization rate UjComprises the following steps:
Uj=αUj-pes+βUj-ram
alpha is more than 0 and less than 1, beta is more than 0 and less than 1, and alpha + p is 1;
definition of TagijAt the current time t, the host pmjSatisfying virtual machine vmiResource requirements of, i.e.
1.3 host availability, availability of a node means the probability of a node working at any time during the entire service time, for any network component i, its availability AiObtained by the following formula:
wherein MTTF represents mean time to failure, MTTR represents mean time to repair, assuming that the value of server availability is known and the availability between servers is independent and independent of each other;
1.4 calculating the power consumption, and in a cloud data center with n running physical hosts, for any physical host pmjE PM, the power consumption at a certain time t is shown by the following formula:
wherein c isjFor static energy consumption marking, fj(t) is time t host pmjCPU frequency, CPU utilization of Uj-pes(t), k is a constant coefficient, i.e. the power consumption is to some extent based on a linear model of the CPU utilization,
1.5 defining virtual machine Placement, VM set by Placement group pkE.g. P, selecting a host in the corresponding physical host set to complete placement mapping, and defining a virtual machine placement matrix M by meeting various constraint conditions in the placement processk[i][j]If M is presentk[i][j]1 denotes the placement group pkPlace virtual machine j on physical host i, otherwise, if M isk[i][j]0 denotes the placement group pkIn, virtual machine j is not placed on physical host i;
the second step is that: setting constraint conditions and optimization targets for virtual machine placement, wherein the process is as follows:
2.1 regarding the virtual machine placement problem in the cloud environment, not only needs to consider meeting the requirements of virtual machine resources, but also needs to consider how to reduce the energy consumption of a data center and efficiently utilize the resources, and in addition, needs to consider the availability problem of the placement request; therefore, the constraints to be considered are: the maximum use number of the server nodes is minimum, the energy consumption is minimum, the load is balanced, and the availability of the placement request is high, and the following constraint conditions are provided:
2.1.1 placing constraints, arbitrary virtual machine vmiUnder the same placing group, the server node can be placed on only one server node;
the constraints represent:
In the same placement group, a single virtual machine is considered to be deployed and operated on only one server node;
2.1.2 resource constraints, for any server nodeIn other words, the consumption of each resource type should not exceed an upper limit, defining a server pmjRespectively, of CPU and memory capacityAndrepresents;
the constraints represent:
The parameter r is a constant coefficient, a part of resources are required to be reserved by the server node to ensure the normal operation of the server node, and r is less than or equal to 1;
2.1.3 reachability constraints, defining a function F (m, n, D) for representing reachability of inter-node communications, for any link (m, n) e.L, if communication delays of points m and n are at most D, the function F (m, n, D) returns 1, otherwise returns 0;
2.2, the virtual machine placement problem has a plurality of optimization targets, and the virtual machine placement problem is optimized and researched in two aspects of availability and energy consumption;
2.2.1 usability optimization
Assuming that the user request consists of a virtual machine between n different VM pairs with associated communication requirements, it is placed on the same server node pmjNot more than once, the usability of the placement cannot be improved because when pmjAt failure, all placement is at pmjWill fail simultaneously, and thus it is desirable to try to get vm toiPlaced on different nodes to increase availability; by HiTo represent placement of virtual machines vmiThe maximum number of nodes, i.e. HiRepresents vmiThe maximum number of server nodes that can be placed; definition ofThe node number of the n virtual machines is H at most;
the availability definition and calculation of virtual machine placement can be divided into three categories: single placement, fully protected placement, and partially protected placement;
2.2.1.1 Single Placement
Single placement means that each virtual machine is placed on only one server node, i.e. H ═ 1, in the case of single placement, if the availability of n server nodes is a respectively1,A2,...,AnK virtual machines (n ≦ k) are placed on the n nodes, then the availability of the virtual machine placement scheme may be used as ApExpressed, defined as follows:
since the request contains k virtual machines, the probability that all the k virtual machines are running needs to be considered when calculating the availability;
2.2.1.2 complete protective Placement
Full protection placement refers to placement of arbitrary virtual machinesAre all placed in group pi(i is more than or equal to 1 and less than or equal to H) are placed on H different nodes; therefore, it can be considered that a full protection placement solution P may be composed of H single placement solutions, and within each single placement solution, placement, resource, and communication reachability constraints should be satisfied between pairs of virtual machines;
availability of a full protection placement solution is the probability that there is at least one placement group working within the life cycle of the service, and the availability is calculated as follows:
2.2.1.3 partial protection Placement
Partial protection placement means presence of virtual machine vmiE.g. VM, placed on less than H different nodes, i.e. two or more placement groups will virtual machine VMiPlaced on the same node and having a certain virtual machine vmjE.g. VM, so that H is greater than 1, and under the condition of placement of partial protection, if a virtual machine is placed on less than H nodes, the virtual machine can be considered to be placed by a plurality of placement groups together; the availability of the server node cannot be directly calculated by the formula in 2.2.1.2 because the availability of the server node where the shared virtual machine is placed is calculated twice; to handle this type of placement, redefine the operator-; suppose there are n nodes pm1,pm2,...,pmnTheir availability is A1,A2,...,An(ii) a For availability of AxNode pmxThe following definition is given for the operator · s:
then according to the formula in the above, defineFor the operation between different sets, the availability of protection placement is calculated by the following formula:
2.2.2 energy consumption optimization
According to the formula in 1.4, during the period T, the physical host pmjTotal energy consumption ofExpressed as:
therefore, the following formula shows that the total energy consumption E of the servers of the data center is within the T periodTFor the sum of the energy consumptions of the various running servers:
the third step: creating a model
Based on the constraint conditions and the optimization target of virtual machine placement given in the second step, a virtual hierarchical structure model VHAM-R based on a Rendervous hash algorithm is established for optimizing and deciding the selection process of the virtual machine to the host, and the steps are as follows:
3.1 initializing a host set PM and a virtual machine set VM, wherein the number of the virtual machines to be placed in the group H at most and the availability set A of host nodes are obtained, if the number n of the host is less than 4, the step 3.2 is carried out, otherwise, the step 3.3 is carried out;
3.2 when the number n of hosts is less than 4, i.e. the number 2 x 2 of the minimum bi-level virtual structure cannot be constructed, defining that the host has a set W of assigned weight scores corresponding to each hosti={wi1,wi2,...,wikWhere k.gtoreq.n defines wijFor virtual machines vmiAt host pmjA weight score of (1), wij=h(vmi,pmj) Wherein the function h (vm, pm) is a hash function agreed in the Renderkvous hash algorithm, and then passes through h (vmi,pmj) Virtual machine vmiIs assigned to a weight wijMaximum host pmjIf the host pmkIs h times that of other hosts, pmkThe equal share is divided into h shares, and the probability of the virtual machine distributed to the host is h times of that of other hosts;
3.3 the number n of the hosts is more than or equal to 4, the cluster of the hosts is divided, firstly, a constant z is selected, namely the number of the hosts in each cluster is z, and the set of the hosts is c-ceiling (n/z), wherein the ceiling function represents dividing n by zRounding up the value to the nearest integer, C0={cpm1,cpm2,...,cpmz},C1={cpmz+1,cpmz+2,...,cpm2zUntil each host belongs to a cluster, each cluster is the lowest node in the virtual hierarchical structure;
the fourth step: the genetic algorithm operation improvement based on the VHAM-R model comprises the following steps:
4.1 for solving the process of placing the virtual machine in the server node, a grouping coding mode based on the host node cluster is provided, P represents an individual for the placing group, CiRepresents a host cluster CiCorresponding to chromosomes, and corresponding to genes on the host computer on each host computer cluster;
4.2, selecting individuals containing good genes in the population through a fitness function set by the selection operation, and taking the energy consumption of the data center and the placement availability of the virtual machines as optimization targets; the process of the selection operation is as follows:
4.2.1 when H1, for each individual, there is a virtual machine placement case, so its availability is calculated as a single placement, i.e. for individual P, its placement availabilityWherein the individual comprises n genes;
4.2.2 when H >1, availability for that H individual needs to be handled in a computationally intensive manner for protected placement;
4.2.2.1 for arbitrary virtual machines vmiE.g. VM, each of which is placed a group piThe virtual machines are placed on H different nodes, i is more than or equal to 1 and less than or equal to H, namely for H individuals, all the virtual machines on each gene are different, and under the condition, the usability of the H placement groups is processed according to a calculation mode of complete protection placement;
4.2.2.2 virtual machine vm if presentiE VM, virtual machine VM in two or more individualsiPlaced on the same gene, in which case the availability of the H placement groups is handled in a computational manner of partial protection placement;
4.2.3 combine data center energy consumption optimization to give the fitness function of the following genetic algorithm:
wherein E isminThe minimum value of the energy consumption of the data center in the T time period is obtained;energy consumption for a single individual; when H ═ 1, availability is calculated in a single placement manner; when H >1, the availability calculations are divided into full protection placements and partial protection placements, where x is the number H of individual groups; for an individual or an individual group x, if the energy consumption of the data center is lower and the availability is higher, the value of the fitness function is larger, and the selected probability of the good genes being inherited to the next generation is higher;
4.3 crossover operation in genetic algorithm is to simulate the mating process between individuals in the biological world, and achieve gene recombination between parents through mating, so that the offspring obtains a new chromosome containing an excellent gene and generates more excellent offspring; the process of the crossover operation is as follows:
4.3.1 first choose two parents to be mated, named X, Y, randomly choose a certain node cluster containing one or more genes in the X parent as a part to be crossed, insert the node cluster, namely all the genes in the node cluster, into the cross point of the Y parent, and then generate new filial generation containing X, Y parent genes;
4.3.2 after completing gene insertion, because the invention uses the chromosome grouping coding mode based on the host cluster, the same host cluster may appear, if the situation appears, the inserted gene is merged into the original host cluster;
4.3.3 if the same two virtual machines exist on different host nodes, temporarily removing the host nodes which previously contain the same virtual machines from the chromosome codes;
4.3.4, the host nodes are temporarily eliminated, and virtual machine nodes which are not deployed by other hosts are possibly included, for the situation, the virtual machines need to be recoded into the host nodes, and genes which meet the constraint conditions and have the lowest energy consumption and the highest availability in the chromosome are selected to be distributed;
4.3.5 if all genes do not meet the requirements, regenerating a new gene segment according to the VHAM-R model, interchanging two parent individuals, and repeating the crossing process to generate a second filial generation individual;
4.4 mutation operation in genetic algorithm, the process of mutation operation is:
4.4.1 determining the individual chromosome gene to be mutated by the mutation function, as shown in the following formula:
wherein U isj-pes、Uj-ramRespectively the CPU and the internal memory utilization ratio of the host,is a set parameter;
4.4.2 selection of fc(j) The smaller gene is deleted, so that the gene with lower utilization rate is deleted every time,
4.4.3 inserting the virtual machine on the gene into other genes by a 4.3 cross operation method.
2. The method for genetic optimization of placement of virtual machines based on VHAM-R model in cloud computing environment according to claim 1, wherein in said step 3.3, the process of clustering the hosts is:
3.3.1 determining virtual leaf node sectors and the depth of a virtual hierarchical structure, selecting the leaf number f of each sub-node sector in the virtual hierarchical structure, wherein f is an integer, and selecting proper f and z can enable the obtained algorithm benefit and load balance degree to be approximate to the expectation; according to the leaf number f of the node sector and the number c of the host clusters, the depth d of the virtual hierarchical structure can be obtained:
fd≥c
wherein d is the smallest positive integer, and d is calculated so that the above formula holds;
3.3.2 numbering each virtual leaf node sector, wherein each sector is uniformly numbered by adopting a natural number from 0, 1, 2.
3.3.3 for a certain virtual machine vmiFor any virtual node s, there is a corresponding weight wis=h(vmi,s),h(vmiS) contains a hash function agreed upon, which passes through h (vm) at each level of the virtual hierarchyiS) calculating the weight of each virtual node, selecting the node with the highest score to continue layering downwards until the real host node cluster Cx to the bottom layer is selected;
3.3.4 virtual machine vmiSelecting a real host node cluster CxLater, when real node selection is carried out, the fact that any real node cluster C exists is assumedxHost node cpm of (1)xz+jAll have a corresponding weight score Wi(xz+j)=H(vmi,cpmxz+j)*Tagi(xz+j)If Tag is false, the value is 0, and if true, the value is 1; wherein the virtual machine vm is usediCpm assigned to hostxz+jThen, H (vm)i,cpmxz+j) For the same period of T, EoldAnd assigning virtual machines vmiReal host node cluster CxRatio of total power consumption of to the host pmxz+jResource utilization rate Uxz+jProduct of the difference with 1 and the corresponding weight constant and sum of host availability and coefficient product:
wherein Exz+jFor the cpm of the host in the T time periodxz+jEnergy consumption of EoldMeans that when no new virtual machine is allocated in the same T time period, the real host node cluster CxEnergy consumption of (2); α, β, γ are weights representing the three; a. thexz+jCpm for the hostxz+jAvailability of (2);
3.3.5 loop through steps 3.3.3-3.3.4 until all virtual machines vmiFinally, the weighting score W is selectedi(xz+j)The highest host node completes the allocation.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201811079838.9A CN109447264B (en) | 2018-09-17 | 2018-09-17 | Virtual machine placement genetic optimization method based on VHAM-R model in cloud computing environment |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201811079838.9A CN109447264B (en) | 2018-09-17 | 2018-09-17 | Virtual machine placement genetic optimization method based on VHAM-R model in cloud computing environment |
Publications (2)
Publication Number | Publication Date |
---|---|
CN109447264A CN109447264A (en) | 2019-03-08 |
CN109447264B true CN109447264B (en) | 2021-11-23 |
Family
ID=65532534
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201811079838.9A Active CN109447264B (en) | 2018-09-17 | 2018-09-17 | Virtual machine placement genetic optimization method based on VHAM-R model in cloud computing environment |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN109447264B (en) |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111124590A (en) * | 2018-10-31 | 2020-05-08 | 千寻位置网络有限公司 | Virtual machine placing method and device and server |
CN110308993B (en) * | 2019-06-27 | 2022-12-13 | 大连理工大学 | Cloud computing resource allocation method based on improved genetic algorithm |
JP2021149129A (en) * | 2020-03-16 | 2021-09-27 | 富士通株式会社 | Fee calculation program and method for calculating fee |
CN112286677B (en) * | 2020-08-11 | 2021-07-16 | 安阳师范学院 | Resource-constrained edge cloud-oriented Internet of things application optimization deployment method |
CN112732408A (en) * | 2021-01-18 | 2021-04-30 | 浪潮云信息技术股份公司 | Method for computing node resource optimization |
Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102984137A (en) * | 2012-11-14 | 2013-03-20 | 江苏南开之星软件技术有限公司 | Multi-target server scheduling method based on multi-target genetic algorithm |
CN103024048A (en) * | 2012-12-17 | 2013-04-03 | 南京邮电大学 | Resources scheduling method under cloud environment |
CN103576829A (en) * | 2012-08-01 | 2014-02-12 | 复旦大学 | Hybrid genetic algorithm based dynamic cloud-computing virtual machine scheduling method |
US9317337B2 (en) * | 2012-04-13 | 2016-04-19 | International Business Machines Corporation | Utilizing software component metadata to provision virtual machines in a networked computing environment |
CN105740051A (en) * | 2016-01-27 | 2016-07-06 | 北京工业大学 | Cloud computing resource scheduling realization method based on improved genetic algorithm |
CN107193640A (en) * | 2017-06-26 | 2017-09-22 | 浙江工业大学 | A kind of virtual machine initial placement strategy process based on Rendezvous hash algorithms |
CN107329815A (en) * | 2017-06-26 | 2017-11-07 | 浙江工业大学 | A kind of cloud task load equalization scheduling method searched for based on BP Tabu |
WO2017192343A1 (en) * | 2016-05-06 | 2017-11-09 | Microsoft Technology Licensing, Llc | Cloud storage platform providing performance-based service level agreements |
US9947236B2 (en) * | 2012-07-10 | 2018-04-17 | Inetwork, Inc. | Apparatus, system, and method for a virtual instruction cloud |
CN108009003A (en) * | 2017-12-01 | 2018-05-08 | 湖南女子学院 | A kind of multiple target virtual machine adaptive location system of selection, distributed cloud system |
-
2018
- 2018-09-17 CN CN201811079838.9A patent/CN109447264B/en active Active
Patent Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9317337B2 (en) * | 2012-04-13 | 2016-04-19 | International Business Machines Corporation | Utilizing software component metadata to provision virtual machines in a networked computing environment |
US9947236B2 (en) * | 2012-07-10 | 2018-04-17 | Inetwork, Inc. | Apparatus, system, and method for a virtual instruction cloud |
CN103576829A (en) * | 2012-08-01 | 2014-02-12 | 复旦大学 | Hybrid genetic algorithm based dynamic cloud-computing virtual machine scheduling method |
CN102984137A (en) * | 2012-11-14 | 2013-03-20 | 江苏南开之星软件技术有限公司 | Multi-target server scheduling method based on multi-target genetic algorithm |
CN103024048A (en) * | 2012-12-17 | 2013-04-03 | 南京邮电大学 | Resources scheduling method under cloud environment |
CN105740051A (en) * | 2016-01-27 | 2016-07-06 | 北京工业大学 | Cloud computing resource scheduling realization method based on improved genetic algorithm |
WO2017192343A1 (en) * | 2016-05-06 | 2017-11-09 | Microsoft Technology Licensing, Llc | Cloud storage platform providing performance-based service level agreements |
CN107193640A (en) * | 2017-06-26 | 2017-09-22 | 浙江工业大学 | A kind of virtual machine initial placement strategy process based on Rendezvous hash algorithms |
CN107329815A (en) * | 2017-06-26 | 2017-11-07 | 浙江工业大学 | A kind of cloud task load equalization scheduling method searched for based on BP Tabu |
CN108009003A (en) * | 2017-12-01 | 2018-05-08 | 湖南女子学院 | A kind of multiple target virtual machine adaptive location system of selection, distributed cloud system |
Non-Patent Citations (4)
Title |
---|
Auto-tuning Performance of MPI Parallel Programs Using Resource Management in Container-Based Virtual Cloud;Hongyi Ma,et al.;《2016 IEEE 9th International Conference on Cloud Computing (CLOUD)》;20170119;全文 * |
Data-Aware Virtual Machine Placement and Rate Allocation in Cloud Environment;Kamran Zamanifar, et al.;《2012 Second International Conference on Advanced Computing & Communication Technologies》;20120315;全文 * |
Virtual Machine Placement in Predictable Computing Clouds;Richard Rauscher,et al.;《2014 IEEE 7th International Conference on Cloud Computing》;20141204;全文 * |
面向多源异构数据的云推送平台的研究与应用;陆佳炜等.;《计算机科学》;20160630;全文 * |
Also Published As
Publication number | Publication date |
---|---|
CN109447264A (en) | 2019-03-08 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN109447264B (en) | Virtual machine placement genetic optimization method based on VHAM-R model in cloud computing environment | |
Iranmanesh et al. | DCHG-TS: a deadline-constrained and cost-effective hybrid genetic algorithm for scientific workflow scheduling in cloud computing | |
Alresheedi et al. | Improved multiobjective salp swarm optimization for virtual machine placement in cloud computing | |
Szabo et al. | Science in the cloud: Allocation and execution of data-intensive scientific workflows | |
Gan et al. | Genetic simulated annealing algorithm for task scheduling based on cloud computing environment | |
Zhao et al. | An energy-aware algorithm for virtual machine placement in cloud computing | |
CN108170530B (en) | Hadoop load balancing task scheduling method based on mixed element heuristic algorithm | |
CN106506657A (en) | One kind distributes method of adjustment based on multiobject cloud computing virtual machine | |
CN108009003A (en) | A kind of multiple target virtual machine adaptive location system of selection, distributed cloud system | |
CN112286677A (en) | Resource-constrained edge cloud-oriented Internet of things application optimization deployment method | |
CN106227599B (en) | The method and system of scheduling of resource in a kind of cloud computing system | |
CN106484512A (en) | The dispatching method of computing unit | |
CN105550033A (en) | Genetic-tabu hybrid algorithm based resource scheduling policy method in private cloud environment | |
CN110008023B (en) | Cloud computing system budget constraint random task scheduling method based on genetic algorithm | |
CN111176784B (en) | Virtual machine integration method based on extreme learning machine and ant colony system | |
Idrissi et al. | A new approach for a better load balancing and a better distribution of resources in cloud computing | |
CN108647771A (en) | The layout method of research-on-research flow data under a kind of mixing cloud environment | |
CN112187535A (en) | Server deployment method and device in fog computing environment | |
Lu et al. | Optimal machine placement based on improved genetic algorithm in cloud computing | |
Yousefipour et al. | Improving the load balancing and dynamic placement of virtual machines in cloud computing using particle swarm optimization algorithm | |
CN104580518A (en) | Load balance control method used for storage system | |
TW201734831A (en) | Method of assigning application to assigned service cluster and device | |
CN113075995A (en) | Virtual machine energy-saving integration method, system and storage medium based on mixed group intelligence | |
CN109343933B (en) | Virtual machine initial placement strategy method based on improved genetic algorithm | |
Fan et al. | Research on improved 2D-BPSO-based VM-container hybrid hierarchical cloud resource scheduling mechanism |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |