CN109447264B - Virtual machine placement genetic optimization method based on VHAM-R model in cloud computing environment - Google Patents

Virtual machine placement genetic optimization method based on VHAM-R model in cloud computing environment Download PDF

Info

Publication number
CN109447264B
CN109447264B CN201811079838.9A CN201811079838A CN109447264B CN 109447264 B CN109447264 B CN 109447264B CN 201811079838 A CN201811079838 A CN 201811079838A CN 109447264 B CN109447264 B CN 109447264B
Authority
CN
China
Prior art keywords
placement
host
virtual machine
availability
node
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201811079838.9A
Other languages
Chinese (zh)
Other versions
CN109447264A (en
Inventor
陆佳炜
赵伟
李�杰
吴涵
肖刚
高燕煦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University of Technology ZJUT
Original Assignee
Zhejiang University of Technology ZJUT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University of Technology ZJUT filed Critical Zhejiang University of Technology ZJUT
Priority to CN201811079838.9A priority Critical patent/CN109447264B/en
Publication of CN109447264A publication Critical patent/CN109447264A/en
Application granted granted Critical
Publication of CN109447264B publication Critical patent/CN109447264B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/12—Computing arrangements based on biological models using genetic models
    • G06N3/126—Evolutionary algorithms, e.g. genetic algorithms or genetic programming
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biophysics (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Physics & Mathematics (AREA)
  • Molecular Biology (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Physiology (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Genetics & Genomics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Multi Processors (AREA)

Abstract

A virtual machine placement genetic optimization method based on a VHAM-R model in a cloud computing environment comprises the following steps: the first step is as follows: the following formalized description is proposed for the virtual machine placement problem, the process is as follows: 1.1 defining a placement environment; 1.2 defining the resource state; 1.3 host availability; 1.4 calculating the electric energy consumption; 1.5 defining virtual machine placement; the second step is that: setting constraint conditions and optimization targets for virtual machine placement; the third step: creating a model: based on the constraint conditions and the optimization target of virtual machine placement given in the second step, establishing a virtual hierarchical structure model VHAM-R based on a Rendervous hash algorithm, and using the virtual hierarchical structure model VHAM-R to optimize and decide the selection process of the virtual machine to the host; the fourth step: and (3) improving the operation of the genetic algorithm based on the VHAM-R model. The invention improves the execution efficiency of the algorithm and the optimization of the finally obtained solution set.

Description

Virtual machine placement genetic optimization method based on VHAM-R model in cloud computing environment
Technical Field
The invention relates to the field of virtual machine placement in a cloud computing environment, in particular to optimization and improvement of virtual machine placement through encoding, selection operation, cross operation and mutation operation of a genetic algorithm on a VHAM-R model.
Background
Cloud computing is derived from distributed computing and grid computing, is a computing mode completely based on the internet, and provides low-cost, high-reliability and scalable computing resources and services for users according to a pay-as-needed mode. The basic idea of cloud computing is to provide physical device support through huge data centers distributed around the world, and provide high-quality computing and storage services for users through the internet at a low price based on a virtualization technology. Virtual Machine Placement (Virtual Machine platform) is a binning problem between cloud data center Virtual machines and physical hosts, an important component of resource management and allocation in a cloud computing environment. The essence of the problem is that the virtual machine is placed on a better physical node through a reasonable distribution method, and meanwhile, the resource requirement and specific constraint condition for running the virtual machine need to be met, so that the problem is an NP-hard problem. The good virtual machine placement strategy can effectively improve the resource utilization rate of each physical host of the cloud data center, reduce the overall energy consumption of the cloud data center, ensure the usability of user requirements and the like.
Z Zhang, CC Hsu and the like take the scale of a data center, the workload of a host and the requirement change of computing resources into consideration, and an energy-saving framework for placing a virtual machine on the host is provided, so that the problems of minimum resource waste and minimum energy consumption of a resource allocation virtual machine are effectively solved. X Li and Z Qian et al investigated the problem of selecting the appropriate physical host to deploy a virtual machine at runtime. As the physical host resources have multidimensional property and the use imbalance of the multidimensional resources can cause resource waste, the multidimensional resource partitioning model is provided for balancing the utilization rate of the multidimensional resources and reducing the number of the running physical hosts, so that the energy consumption of the data center is reduced. The domestic Liqiang et al provides a model based on long-term load performance for the virtual machine placement problem in the cloud computing environment, and effectively reduces the number of used physical host nodes by combining a genetic algorithm of multi-objective optimization.
The Genetic Algorithm (GA) is proposed by Holland in 1975 by referring to natural selection of the biological evolution theory and the biological evolution process in genetics, and is an algorithm for finding the optimal solution of the NP-hard problem by simulating the biological evolution process. Compared with the common virtual machine Placement algorithm under cloud computing based on ANSYS cluster finite element analysis, the virtual machine Placement algorithm comprises the classic algorithms such as First In First Service (First In First Service), First adaptive algorithm (Fist discovery), First adaptive descent algorithm (Fist First discovery), Best adaptive algorithm (Best Fit), Best adaptive descent algorithm (Best Fit), Greedy Placement algorithm (Greedy Placement), and the like, and the genetic algorithm can better find the global optimal solution.
Disclosure of Invention
In order to place the group of virtual machine placement requests on the server nodes, the optimization target of high virtual machine placement availability with fewer working hosts and low energy consumption of the data center is realized on the premise of meeting the constraint conditions of placement, resources, communication accessibility and the like. The invention provides an improved genetic algorithm for processing the virtual machine placement problem, and the encoding mode, selection, mating process and variation mode of the virtual machine placement problem are improved based on a VHAM-R model, so that the execution efficiency of the algorithm is improved, and the finally obtained solution set is optimized.
The invention provides the following technical scheme for solving the technical problems:
a virtual machine placement genetic optimization method based on a VHAM-R model in a cloud computing environment comprises the following steps:
the first step is as follows: the following formalized description is proposed for the virtual machine placement problem, the process is as follows:
1.1 define the placement environment, and the data center has a physical host set PM ═ PM1,pm2,…,pmnN, the number of hosts is n, and a virtual machine set VM ═ VM needs to be placed1,vm2,…,vmmAnd where the number of virtual machines is m, and assuming that the number of virtual machines m is greater than or equal to the number of hosts n, defining a set of virtual machine placement groups P ═ { P ═ P1,p2,…,phH is the number of the placing groups;
1.2 defining resource states, vm for a given virtual machineiDefinition of
Figure BDA0001801645280000021
For virtual machines vmiThe required resources of the CPU are used,
Figure BDA0001801645280000022
for virtual machines vmiRequired memory resource, Vi-pesFor virtual machines vmiCPU utilization of Wi-ramFor virtual machines vmiMemory utilization of, for a given host pmjDefinition of
Figure BDA0001801645280000023
Is a main engine pmjThe current resources of the CPU are free from the CPU,
Figure BDA0001801645280000024
is a main engine pmjFree resources of memory, Uj-pesIs a main engine pmjCPU utilization, Uj-ramIs a main engine pmjThe memory utilization rate of (1) then defines the host pmjResource utilization rate UjComprises the following steps:
Uj＝αUj-pes+βUj-ram
0< α <1,0< β <1, and α + β ═ 1;
definition of TagijAt the current time t, the host pmjWhether or not to satisfy vm of virtual machineiResource requirements of, i.e.
Figure BDA0001801645280000025
1.3 host availability, availability of a node means the probability of a node working at any time during the entire service time, for any network component i, its availability AiObtained by the following formula:
Figure BDA0001801645280000031
wherein MTTF represents mean time to failure, MTTR represents mean time to repair, assuming that the value of server availability is known and the availability between servers is independent and independent of each other;
1.4 calculating the power consumption, and in a cloud data center with n running physical hosts, for any physical host pmjE PM, the power consumption at a certain time t is shown by the following formula:
Figure BDA0001801645280000032
wherein c isjFor static energy consumption marking, fj(t) is time t host pmjCPU frequency, CPU utilization of Uj-pes(t), k is a constant coefficient, i.e. the power consumption is to some extent based on a linear model of the CPU utilization,
1.5 defining virtual machine Placement, VM set by Placement group pkE.g. P, selecting a host in the corresponding physical host set to complete placement mapping, and defining a virtual machine placement matrix M by meeting various constraint conditions in the placement process as much as possiblek[i][j]If M is presentk[i][j]1 denotes the placement group pkPlace virtual machine j on physical host i, otherwise, if M isk[i][j]0 denotes the placement group pkIn (1), virtual machine j is not placed on physical host i,
the second step is that: setting constraint conditions and optimization targets for virtual machine placement, wherein the process is as follows:
2.1 regarding the virtual machine placement problem in the cloud environment, not only needs to consider meeting the requirements of virtual machine resources, but also needs to consider how to reduce the energy consumption of a data center and efficiently utilize the resources, and in addition, needs to consider the availability problem of the placement request; therefore, the constraints to be considered are: the maximum use number of the server nodes is minimum, the energy consumption is minimum, the load is balanced, and the availability of the placement request is high, and the following constraint conditions are provided:
2.1.1 placing constraints, arbitrary virtual machine vmiUnder the same placing group, the two can be placed in one and only oneA server node;
the constraints represent:
for the
Figure BDA0001801645280000033
In which a group p is placedk∈P；
In the same placement group, a single virtual machine is considered to be deployed and operated on only one server node;
2.1.2 resource constraints, for any server node, the consumption of each resource type should not exceed the upper limit, defining server pmjRespectively, of CPU and memory capacity
Figure BDA0001801645280000034
And
Figure BDA0001801645280000035
represents;
the constraints represent:
for the
Figure BDA0001801645280000041
Is provided with
Figure BDA0001801645280000042
Figure BDA0001801645280000043
The parameter is a constant coefficient, a part of resources are required to be reserved by the server node to ensure the normal operation of the server node, r is less than or equal to 1, and the normal value is 0.8;
2.1.3 reachability constraints, defining a function F (m, n, D) for representing reachability of inter-node communications, for any link (m, n) e.L, if communication delays of points m and n are at most D, the function F (m, n, D) returns 1, otherwise returns 0;
2.2, the virtual machine placement problem has a plurality of optimization targets, and the virtual machine placement problem is optimized and researched in two aspects of availability and energy consumption;
2.2.1 usability optimization
Assuming that the user request consists of a virtual machine between n different VM pairs with associated communication requirements, it is placed on the same server node pmjNot more than once, the usability of the placement cannot be improved because when pmjAt failure, all placement is at pmjWill fail simultaneously, and thus it is desirable to try to get vm toiPlaced on different nodes to increase availability; by HiTo represent placement of virtual machines vmiThe maximum number of nodes, i.e. HiRepresents vmiThe maximum number of server nodes that can be placed; definition of
Figure BDA0001801645280000044
The node number of the n virtual machines is H at most;
the availability definition and calculation of virtual machine placement can be divided into three categories: single placement, fully protected placement, and partially protected placement;
2.2.1.1 Single Placement
Single placement means that each virtual machine is placed on only one server node, i.e. H ═ 1, in the case of single placement, if the availability of n server nodes is a respectively1,A2,…,AnK virtual machines (n ≦ k) are placed on the n nodes, then the availability of the virtual machine placement scheme may be used as ApExpressed, defined as follows:
Figure BDA0001801645280000045
since the request contains k virtual machines, the probability that all the k virtual machines are running needs to be considered when calculating the availability;
2.2.1.2 complete protective Placement
Full protection placement refers to placement of arbitrary virtual machines
Figure BDA0001801645280000046
Are all placed in group pi(i is more than or equal to 1 and less than or equal to H) are placed on H different nodes; thus, a complete protection can be consideredThe placement plan P can be composed of H single placement plans, and in each single placement plan, the virtual machine pair should satisfy placement, resource and communication accessibility constraints;
availability of a full protection placement solution is the probability that there is at least one placement group working within the life cycle of the service, and the availability is calculated as follows:
Figure BDA0001801645280000051
2.2.1.3 partial protection Placement
Partial protection placement means presence of virtual machine vmiE.g. VM, placed on less than H different nodes, i.e. two or more placement groups will virtual machine VMiPlaced on the same node and having a certain virtual machine vmjE.g. VM, so that H is greater than 1, and under the condition of placement of partial protection, if a virtual machine is placed on less than H nodes, the virtual machine can be considered to be placed by a plurality of placement groups together; the availability of the server node cannot be directly calculated by the formula in 2.2.1.2 because the availability of the server node where the shared virtual machine is placed is calculated twice; redefining the operator to handle the placement; suppose there are n nodes pm1,pm2,…,pmnTheir availability is A1,A2,…,An(ii) a For availability of AxNode pmxThe following definitions are given for the operators:
Figure BDA0001801645280000052
then according to the formula in the above, define
Figure BDA0001801645280000056
For operations between different sets, the availability of protection placement is calculated by the following formula:
Figure BDA0001801645280000053
2.2.2 energy consumption optimization
According to the formula in 1.4, during the period T, the physical host pmjTotal energy consumption of
Figure BDA0001801645280000054
Expressed as:
Figure BDA0001801645280000055
therefore, the following formula shows that the total energy consumption E of the servers of the data center is within the T periodTFor the sum of the energy consumptions of the various running servers:
Figure BDA0001801645280000061
the third step: creating a model
Based on the constraint conditions and the optimization target of virtual machine placement given in the second step, a virtual hierarchical structure model (VHAM-R) based on a Rendervous hash algorithm is established for optimizing and deciding the selection process of the virtual machine to the host, and the steps are as follows:
3.1 initializing a host set PM and a virtual machine set VM, wherein the number of the virtual machines to be placed in the group H at most and the availability set A of host nodes are obtained, if the number n of the host is less than 4, the step 3.2 is carried out, otherwise, the step 3.3 is carried out;
3.2 when the number n of hosts is less than 4, i.e. the number 2 x 2 of the minimum bi-level virtual structure cannot be constructed, defining that the host has a set W of assigned weight scores corresponding to each hosti＝{wi1,wi2,…,wikWhere k.gtoreq.n defines wijFor virtual machines vmiAt host pmjA weight score of (1), wij＝h(vmi,pmj) Wherein the function h (vm, pm) is a hash function agreed in the Renderkvous hash algorithm, and then passes through h (vmi,pmj) Virtual machine vmiIs assigned to a weight wijMaximum host pmjIf the host pmkIs h times that of other hosts, pmkThe equal share is divided into h shares, and the probability of the virtual machine distributed to the host is h times of that of other hosts;
3.3 the number n of the hosts is more than or equal to 4, for the host cluster division, firstly, a constant z is selected, namely the number of the hosts in each cluster is z, and the host set is C-ceiling (n/z), wherein the ceiling function represents that the value of dividing n by z is rounded up to the nearest integer, C0＝{cpm1，cpm2，…，cpmz}，C1＝{cpmz+1，cpmz+2，…， cpm2z}, … until each host belongs to a cluster, each cluster being the lowest node in the virtual hierarchy;
the fourth step: the genetic algorithm operation improvement based on the VHAM-R model comprises the following steps:
4.1 for solving the process of placing the virtual machine in the server node, a grouping coding mode based on the host node cluster is provided, P represents an individual for the placing group, CiRepresenting that the host clusters correspond to chromosomes and the host on each host cluster corresponds to a gene;
4.2, selecting individuals containing good genes in the population through a fitness function set by the selection operation, and taking the energy consumption of the data center and the placement availability of the virtual machines as optimization targets;
4.3 crossover operation in genetic algorithm is to simulate the mating process between individuals in the biological world, and achieve gene recombination between parents through mating, so that the offspring obtains a new chromosome containing an excellent gene and generates more excellent offspring;
4.4 mutation operations in genetic algorithms.
Further, in step 3.3, the process of dividing the host cluster includes:
3.3.1 determining virtual leaf node sectors and the depth of a virtual hierarchical structure, selecting the leaf number f of each sub-node sector in the virtual hierarchical structure, wherein f is an integer, and selecting proper f and z can enable the obtained algorithm benefit, load balance degree and the like to be approximate to expectations; according to the leaf number f of the node sector and the number z of the host clusters, the depth d of the virtual hierarchical structure can be obtained:
fd≥c
wherein d is the smallest positive integer, and d is calculated so that the above formula holds;
3.3.2 numbering each virtual leaf node sector, numbering each sector uniformly by adopting natural numbers respectively, from 0, 1, 2, …, f-1;
3.3.3 for a certain virtual machine vmiFor any virtual node s, there is a corresponding weight wis＝h(vmi,s)，h(vmiS) includes a hash function of a convention, such as hash32, hash64, etc., that passes through h (vm) at each level of the virtual hierarchyiS) calculating the weight of each virtual node, selecting the node with the highest score to continue layering downwards until the real host node cluster Cx to the bottom layer is selected;
3.3.4 virtual machine vmiSelecting a real host node cluster CxLater, when real node selection is carried out, the fact that any real node cluster C exists is assumedxHost node cpm of (1)xz+jAll have a corresponding weight score Wi(xz+j)＝H(vmi,cpmxz+j)*Tagi(xz+j)If Tag is false, the value is 0, and if true, the value is 1; wherein the virtual machine vm is usediCpm assigned to hostxz+jThen, H (vm)i,cpmxz+j) For the same period of T, EoldAnd assigning virtual machines vmiReal host node cluster CxRatio of total power consumption of to the host pmxz+jResource utilization rate Uxz+jProduct of the difference with 1 and the corresponding weight constant and sum of host availability and coefficient product:
Figure BDA0001801645280000071
wherein Exz+jFor the cpm of the host in the T time periodxz+jEnergy consumption of EoldMeans that in the same T time period, when no new virtual machine is allocated, the virtual machine is realHost node cluster CxEnergy consumption of (2); α, β, γ are weights representing the three; a. thexz+jCpm for the hostxz+jAvailability of (2);
3.3.5 loop through steps 3.3.3-3.3.4 until all virtual machines vmiFinally, the weighting score W is selectedi(xz+j)The highest host node completes the allocation.
Still further, in step 4.2, the process of selecting operation is as follows:
4.2.1 when H1, for each individual, there is a virtual machine placement case, so its availability is calculated as a single placement, i.e. for individual P, its placement availability
Figure BDA0001801645280000081
Wherein the individual comprises n genes;
4.2.2 when H >1, availability for that H individual needs to be handled in a computationally intensive manner for protected placement;
4.2.2.1 for arbitrary virtual machines vmiE.g. VM, each of which is placed a group piThe virtual machines are placed on H different nodes, i is more than or equal to 1 and less than or equal to H, namely for H individuals, all the virtual machines on each gene are different, and under the condition, the usability of the H placement groups is processed according to a calculation mode of complete protection placement;
4.2.2.2 virtual machine vm if presentiE VM, virtual machine VM in two or more individualsiPlaced on the same gene, in which case the availability of the H placement groups is handled in a computational manner of partial protection placement;
4.2.3 combine data center energy consumption optimization to give the fitness function of the following genetic algorithm:
Figure BDA0001801645280000082
wherein E isminThe minimum value of the energy consumption of the data center in the T time period is obtained;
Figure BDA0001801645280000083
energy consumption for a single individual; when H ═ 1, availability is calculated in a single placement manner; when H is present>1, the availability calculation is divided into full protection placement and partial protection placement, wherein x is an individual group with the number of H; for an individual or an individual group x, if the lower the energy consumption of the data center and the higher the availability, the larger the value of the fitness function is, and the higher the selected probability that the good genes are inherited to the next generation is.
Furthermore, in step 4.3, the process of the interleaving operation is as follows:
4.3.1 first choose two parents to be mated, named X, Y, randomly choose a certain node cluster containing one or more genes in the X parent as a part to be crossed, insert the node cluster, namely all the genes in the node cluster, into the cross point of the Y parent, and then generate new filial generation containing X, Y parent genes;
4.3.2 after completing gene insertion, because the invention uses the chromosome grouping coding mode based on the host cluster, the same host cluster may appear, if the situation appears, the inserted gene is merged into the original host cluster;
4.3.3 if the same two virtual machines exist on different host nodes, temporarily removing the host nodes which previously contain the same virtual machines from the chromosome codes;
4.3.4, the host nodes are temporarily eliminated, and virtual machine nodes which are not deployed by other hosts are possibly included, for the case, the virtual machines need to be recoded into the host nodes, and the genes which meet the constraint conditions and have the lowest energy consumption and the highest availability in the chromosome are preferably selected to complete the distribution;
4.3.5 if all genes do not meet the requirements, regenerating a new gene segment according to the VHAM-R model, interchanging two parent individuals, and repeating the crossing process to generate a second filial generation individual.
In the step 4.4, the mutation process includes:
4.4.1 determining the individual chromosome gene to be mutated by the mutation function, as shown in the following formula:
Figure BDA0001801645280000091
wherein U isj-pes、Uj-ramRespectively the CPU and the internal memory utilization ratio of the host,
Figure BDA0001801645280000092
is a set parameter;
4.4.2 selection of fc(j) The smaller gene is deleted, so that the gene with lower utilization rate is deleted every time,
4.4.3 inserting the virtual machine on the gene into other genes by a 4.3 cross operation method.
The invention has the beneficial effects that: and improving the encoding mode, selection, mating process and variation mode of the VHAM-R model based on the VHAM-R model, thereby improving the execution efficiency of the algorithm and the optimization of the finally obtained solution set.
Drawings
FIG. 1 is a VHAM-R model of the invention.
FIG. 2 shows the host cluster-based chromosomal coding of the present invention.
FIG. 3 is a schematic diagram of the crossover operation of the improved genetic algorithm of the present invention.
FIG. 4 is a schematic diagram of the variant operation of the improved genetic algorithm of the present invention.
Detailed Description
The invention is further described below with reference to the accompanying drawings.
Referring to fig. 1 to 4, a virtual machine placement genetic optimization method based on a VHAM-R model in a cloud computing environment includes the following steps:
the first step is as follows: the following formalized description is proposed for the virtual machine placement problem, the process is as follows:
1.1 define the placement environment, and the data center has a physical host set PM ═ PM1,pm2,…,pmnN, the number of hosts is n, and a virtual machine set VM ═ VM needs to be placed1,vm2,…,vmmAnd where the number of virtual machines is m, and assuming that the number of virtual machines m is greater than or equal to the number of hosts n, defining a set of virtual machine placement groups P ═ { P ═ P1,p2,…,phH is the number of placed groups.
1.2 defining resource states, vm for a given virtual machineiDefinition of
Figure BDA0001801645280000101
For virtual machines vmiThe required resources of the CPU are used,
Figure BDA0001801645280000102
for virtual machines vmiRequired memory resource, Vi-pesFor virtual machines vmiCPU utilization of Wi-ramFor virtual machines vmiThe memory utilization ratio. For a given host pmjDefinition of
Figure BDA0001801645280000103
Is a main engine pmjThe current resources of the CPU are free from the CPU,
Figure BDA0001801645280000104
is a main engine pmjFree resources of memory, Uj-pesIs a main engine pmjCPU utilization, Uj-ramIs a main engine pmjThe memory utilization rate of (1) then defines the host pmjResource utilization rate UjComprises the following steps:
Uj＝αUj-pes+βUj-ram
0< α <1,0< β <1, and α + β ═ 1.
Definition of TagijAt the current time t, the host pmjWhether or not to satisfy vm of virtual machineiResource requirements of, i.e.
Figure BDA0001801645280000105
1.3 host availability, availability of a node means the probability of a node working at any time during the whole service time, for any networkNetwork component i, availability A thereofiCan be calculated by the following formula:
Figure BDA0001801645280000106
wherein MTTF represents mean time to failure, MTTR represents mean time to repair, assuming that the value of server availability is known and the availability between servers is independent and independent of each other;
1.4 calculating the power consumption, and in a cloud data center with n running physical hosts, for any physical host pmjE PM, the power consumption at a certain time t is shown by the following formula:
Figure BDA0001801645280000107
wherein c isjFor static energy consumption marking, fj(t) is time t host pmjCPU frequency, CPU utilization of Uj-pes(t), k is a constant coefficient, namely the power consumption is a linear model based on the CPU utilization rate to a certain extent;
1.5 defining virtual machine Placement, VM set by Placement group pkE, selecting a host in a corresponding physical host set to complete placement mapping, wherein the host belongs to P, and various constraint conditions in the placement process need to be met as much as possible; defining a virtual machine placement matrix Mk[i][j]If M is presentk[i][j]1 denotes the placement group pkPlace virtual machine j on physical host i, otherwise, if M isk[i][j]0 denotes the placement group pkIn (1), virtual machine j is not placed on physical host i.
The second step is that: setting constraint conditions and optimization targets for virtual machine placement, wherein the process is as follows:
2.1 regarding the virtual machine placement problem in the cloud environment, not only needs to consider meeting the requirements of virtual machine resources, but also needs to consider how to reduce the energy consumption of a data center and efficiently utilize the resources, and in addition, needs to consider the availability problem of the placement request; therefore, constraints that need to be considered are as long as: the maximum use number of the server nodes is minimum, the energy consumption is minimum, the load is balanced, and the availability of placing the request is high; for the above points, the following constraints are proposed:
2.1.1 placing constraints, arbitrary virtual machine vmiUnder the same placement group, it can be and can only be placed on one server node.
The constraints represent:
for the
Figure BDA0001801645280000111
In which a group p is placedk∈P；
Description of the drawings: generally, in the same placement group, a single virtual machine is considered to be deployed and operated on only one server node;
2.1.2 resource constraint, for any server node, the consumption of each resource type should not exceed the upper limit, and the server resources are generally divided into CPU, memory, network bandwidth, disk resources, etc.; considering the resource condition of CPU and memory, defining server pmjRespectively, of CPU and memory capacity
Figure BDA0001801645280000112
And
Figure BDA0001801645280000113
and (4) showing.
The constraints represent:
for the
Figure BDA0001801645280000114
Is provided with
Figure BDA0001801645280000115
Figure BDA0001801645280000116
Description of the drawings: wherein, the parameter r (less than or equal to 1) is a constant coefficient, a part of resources need to be reserved by the server node to ensure the normal operation of the server node, and the normal value is 0.8;
2.1.3 reachability constraints, defining a function F (m, n, D) for representing reachability of inter-node communications, for any link (m, n) e.L, if communication delays of points m and n are at most D, the function F (m, n, D) returns 1, otherwise returns 0;
2.2 the optimization targets of the virtual machine placement problem are numerous, and typically comprise energy consumption optimization, network flow optimization, resource allocation optimization, availability optimization, performance optimization and the like; selecting two aspects of availability and energy consumption to carry out optimization research on the virtual machine placement problem;
2.2.1 usability optimization
Suppose a user request consists of a virtual machine between n different VM pairs with associated communication requirements (considering communication reachability), which is placed on the same server node (say pm)j) Not more than once, the usability of the placement cannot be improved because when pmjAt failure, all placement is at pmjThe virtual machines on will fail at the same time; therefore, it is necessary to try to make vm as small as possibleiPlaced on different nodes to increase availability. By HiTo represent placement of virtual machines vmiThe maximum number of nodes, i.e. HiRepresents vmiMaximum number of server nodes that can be placed, define
Figure BDA0001801645280000121
The node number of the n virtual machines is H at most;
the availability definition and calculation of virtual machine placement can be divided into three categories: single placement, fully protected placement, partially protected placement;
2.2.1.1 Single Placement
Single placement means that each virtual machine is placed on only one server node, i.e. H ═ 1, in the case of single placement, if the availability of n server nodes is a respectively1,A2,…,AnK virtual machines (n ≦ k) are placed on the n nodes, then the availability of the virtual machine placement scheme may be used as ApExpressed, defined as follows:
Figure BDA0001801645280000122
since the request contains k virtual machines, the probability that all the k virtual machines are running needs to be considered when calculating the availability;
2.2.1.2 complete protective Placement
Full protection placement refers to placement of arbitrary virtual machines
Figure BDA0001801645280000123
Are all placed in group pi(1. ltoreq. i.ltoreq.H) are placed on H different nodes. Therefore, it can be considered that a full protection placement solution P may be composed of H single placement solutions, and within each single placement solution, placement, resource, and communication reachability constraints should be satisfied between pairs of virtual machines;
availability of a full protection placement solution is the probability that there is at least one placement group working within the life cycle of the service, and the availability is calculated as follows:
Figure BDA0001801645280000131
2.2.1.3 partial protection Placement
Partial protection placement means presence of virtual machine vmiE.g. VM, placed on less than H different nodes (i.e. two or more placement groups will virtual machine VM)iPlaced on the same node) and there is a certain virtual machine vmjBelongs to VM, so that H is more than 1; under the condition of partial protection placement, if one virtual machine is placed on less than H nodes, the virtual machine can be considered to be placed together by a plurality of placement groups; its availability cannot be calculated directly by the formula in 2.2.1.2 because the availability of the server node where the shared virtual machine is placed would be calculated twice. To handle this type of placement, operators are redefined. Suppose there are n nodes pm1,pm2,…,pmnTheir availability is A1,A2,…,An(ii) a For availability of AxNode pmxThe following definitions are given for the operators:
Figure BDA0001801645280000132
then according to the above formula, define
Figure BDA0001801645280000136
For operations between different sets, the availability of partial protection placement can be calculated by the following formula:
Figure BDA0001801645280000133
2.2.2 energy consumption optimization
In a cloud computing environment, energy consumption generated by a data center is mainly energy consumption of various devices, including a server, a storage device, a network communication device and the like; the server accounts for the vast majority of energy consumption, the virtual machine placement is optimized from the energy consumption perspective, and a certain optimization target can be achieved by directly or indirectly reducing the starting operation number of the server mainly by reducing the energy consumption of server equipment;
according to the formula in 1.4, during the period T, the physical host pmjTotal energy consumption of
Figure BDA0001801645280000134
Expressed as:
Figure BDA0001801645280000135
therefore, the following formula shows that the total server energy consumption E of the data center is obtained in the T periodTThe sum of the energy consumption of each running server;
Figure BDA0001801645280000141
the third step: creating a model
Based on the constraint conditions and the optimization target of virtual machine placement given in the second step, a virtual hierarchical structure model (VHAM-R) based on a Rendervous hash algorithm is established for optimizing and deciding the selection process of the virtual machine to the host, and the basic idea of the Rendervous hash algorithm is that for each site SjAnd each object thereof OiThe corresponding weight can be calculated by the appointed hash function, and for each site SjSelecting the object O with the greatest weightmAnd is combined with OmTo station Sj；
3.1 initializing a host set PM, a virtual machine set VM, the maximum number of groups H placed by the virtual machines and an availability set A of host nodes, and entering a step 3.2 if the number n of the host machines is less than 4, or entering a step 3.3 if the number n of the host machines is not more than 4. Referring to fig. 1, the number of hosts n in the figure is 108, and step 3.3 is entered;
3.2 when the number n of hosts is less than 4, i.e. the number 2 x 2 of the minimum bi-level virtual structure cannot be constructed, defining that the host has a set W of assigned weight scores corresponding to each hosti＝{wi1,wi2,…,wik(where k.gtoreq.n). Definition of wijFor virtual machines vmiAt host pmjA weight score of (1), wij＝h(vmi,pmj). The function h (vm, pm) includes a predetermined hash function, such as hash32, hash64, etc. . Wherein the function h (vm, pm) is a hash function appointed in the Renderzwaus hash algorithm, and then passes through h (vm)i,pmj) Function-to-virtual machine vmiIs assigned to a weight wijMaximum host pmj. If the host pmkIs h times that of other hosts, pmkIs divided equally into h portions, e.g. pmk1,pmk2,…,pmkh. Obviously, the probability of the virtual machine being allocated to the host is h times that of other hosts;
3.3 the number n of the host computers is more than or equal to 4, and the cluster groups of the host computers are divided. First, a constant z is selected, i.e. the number of hosts in each cluster is z, which is known as 4 in fig. 1. The host set is changed to ceil (108/4) to 27 according to c (ceil (n/z) ═ ceil (108/4) (where the ceil function indicates that the value of n divided by z is upRounded to the nearest integer). C0＝{cpm1， cpm2，…，cpmz}，C1＝{cpmz+1，cpmz+2，…，cpm2zAnd … until each host belongs to a cluster. Each cluster is a bottommost node in the virtual hierarchical structure;
3.3.1 virtual leaf node sectors and virtual hierarchy depth determination. The leaf number f of each child node sector in the virtual hierarchical structure is selected, wherein f is a one-bit integer generally, and the selection of proper f and z can make the obtained algorithm benefit, load balance degree and the like closer to the expectation. As shown in fig. 1, the leaf number of each child node sector is 3, and according to the leaf number of the node sector 3 and the number of host clusters 27, the depth h of the virtual hierarchical structure can be obtained:
fd≥c
wherein d is the minimum positive integer, and d is solved to be 3;
3.3.2 each virtual leaf node sector number. Generally, each sector is numbered uniformly by adopting natural numbers, namely 0, 1, 2, … and f-1;
3.3.3 for a certain virtual machine vmiFor any virtual node s, there is a corresponding weight wis＝h(vmi,s)，h(vmiS) includes a hash function of a convention, such as hash32, hash64, etc., which may be h (vm) at each level of leaf sectors of the virtual hierarchyiS) calculate the virtual node weights, as in FIG. 1, starting from the root node because h (vm)i,0)>max{h(vmi,1),h(vmi2), thus selecting (0)1The node continues to go down; in the second level of three child nodes, because of h (vm)i,00)>max{h(vmi,01),h(vmi02) } thus select (00)1The node continues to go down; in the third layer of three child nodes, because h (vm)i,000)>max{h(vmi,001),h(vmi002) }, thus selecting (000)1Nodes, i.e. C0The true position is a node cluster, and then the next step of selection is carried out;
3.3.4 virtual machine vmiSelecting a real host node cluster C0Then, inWhen selecting the real nodes, the method assumes that the real nodes are randomly in the real node cluster C0Host node cpm of (1)j(1. ltoreq. j. ltoreq.z), all have a corresponding weight score Wij＝H(vmi,cpmj)*Tagij(if Tag is false, it is 0, and if true, it is 1). Wherein the virtual machine vm is usediCpm assigned to hostjThen, H (vm)i,cpmj) For the same period of T, EoldAnd assigning virtual machines vmiReal host node cluster C0Ratio of total power consumption of to the host pmjResource utilization rate UjProduct of the difference with 1 and the corresponding weight constant and sum of host availability and coefficient product:
Figure BDA0001801645280000151
wherein EjFor the cpm of the host in the T time periodjEnergy consumption of EoldMeans that when no new virtual machine is allocated in the same T time period, the real host node cluster C0Energy consumption of (2). Alpha, beta, and gamma are weights representing the three, AjIs a main unit pmjHost availability. The 2 nodes in fig. 1 are the nodes selected finally;
3.3.5 loop through steps 3.3.3-3.3.4 until all virtual machines vmiFinally, the weighting score W is selectedi(xz+j)The highest host node completes the distribution;
the fourth step: genetic algorithm operation improvement based on VHAM-R model
An improved genetic algorithm is provided for processing the virtual machine placement problem, and the encoding mode, the selection, the mating process and the variation mode of the virtual machine placement problem are improved based on a VHAM-R model, so that the execution efficiency of the algorithm and the optimization of a finally obtained solution set are improved;
4.1 the coding mode of the chromosome has important influence on the search effect and the algorithm efficiency of the genetic algorithm, the realization process of the coding is the process of mapping the problem solution to the chromosome, and the invention provides the process of solving the virtual machine and placing the virtual machine to the server node based on the hostAnd (4) a grouping coding mode of the node cluster. P represents a placing group, CiIt means that the host clusters correspond to chromosomes, and the host on each host cluster corresponds to genes. FIG. 2 is a specific example of chromosomal encoding;
the data center in fig. 2 contains two placement groups p1 and p2, the placement request contains 8 virtual machines and a plurality of physical hosts, each placement group is composed of a plurality of host node clusters, each host node cluster has different numbers of host nodes, and each host node has different numbers of virtual machines. As in the left half of the figure, containing host clusters c1, c2, c4, host node E in host cluster c1, host node F in host cluster c2, and host node D in host cluster c 4. Virtual machines 2, 5, 4 are placed in host node E, virtual machines 3, 6, 7 are placed in host node F, and virtual machines 1, 8 are placed in host node D, the placement set corresponding to 3 chromosome basis factors, i.e., chromosome EFD. As in the right half of the figure, containing host clusters C0, C3, C5, host node A, B in host cluster C0, host node G in host cluster C3, and host node C in host cluster C5. Virtual machines 7, 6 are placed in host node a, virtual machines 2, 4 are placed in host node B, virtual machines 1, 5 are placed in host node G, and virtual machines 3, 8 are placed in host node C, the placement group corresponding to a chromosome base factor of 4, i.e., chromosome ABGC. Changing an operation object from a single virtual machine to a host cluster containing a virtual machine group by using a chromosome grouping coding mode based on the host cluster;
4.2 the selection operation of the genetic algorithm is to simulate the excellence and the disadvantage of organisms in the nature, and the selection operation can select individuals with higher adaptability and ensure that better genes can be smoothly inherited to the next generation;
selecting individuals containing excellent genes in the population through a fitness function set by the selection operation, and taking the energy consumption of a data center and the placement availability of a virtual machine as optimization targets;
4.2.1 when H1, for each individual, there is a virtual machine placement case, so its availability is calculated as a single placement, i.e. for individual p, its placement availability
Figure BDA0001801645280000171
Wherein the individual comprises n genes (host nodes);
4.2.2 when H >1, the availability for that H individuals (placement group) needs to be handled in a way that protected placement is calculated;
4.2.2.1 for arbitrary virtual machines vmiE.g. VM, each of which is placed a group pi(i is more than or equal to 1 and less than or equal to H) are placed on H different nodes, namely for H individuals, all virtual machines on each gene are different, and in this case, the availability of the H placement groups is processed according to a calculation mode of complete protection placement;
4.2.2.2 virtual machine vm if presentiE.g. VM, two or more individuals (placement group), virtual machine VMiPlaced on the same gene, in which case the availability of the H placement groups is handled in a computational manner of partial protection placement;
4.2.3 combine data center energy consumption optimization to give the fitness function of the following genetic algorithm:
Figure BDA0001801645280000172
wherein E isminThe minimum value of the energy consumption of the data center in the T time period is obtained;
Figure BDA0001801645280000173
energy consumption for a single individual; when H ═ 1, availability is calculated in a single placement manner; when H is present>1, the availability calculations are divided into full protection placement and partial protection placement, where x is the number H of individual clusters. For an individual or an individual group x, if the energy consumption of the data center is lower and the availability is higher, the value of the fitness function is larger, and the selected probability of the good genes being inherited to the next generation is higher;
4.3 crossover operation in genetic algorithm is to simulate the process of mating between individuals in the biological world, and achieve recombination of genes between parents through mating, so that offspring can obtain new chromosomes containing excellent genes to generate more excellent offspring, which is specifically described with reference to fig. 3;
4.3.1 first pick two parents to be mated, named X, Y in fig. 3, randomly pick a certain node cluster containing one or more genes in the X parent as a part to be crossed, insert the node cluster, i.e. all genes in it, into the Y parent cross point, and at this time, generate a new offspring containing X, Y parent genes;
4.3.2 after completing gene insertion, because the invention uses the chromosome grouping coding mode based on the host cluster, the same host cluster may appear, if the situation appears, the inserted gene is merged into the original host cluster;
4.3.3 if the same two virtual machines exist on different host nodes, temporarily removing the host nodes which previously contain the same virtual machines from the chromosome codes, wherein virtual machines with repeated numbers appear on B, E and G in the figure 3, and removing the host machines B and G;
4.3.4, are temporarily dropped from the host node, may contain virtual machine nodes that are not deployed by other hosts,
in FIG. 3, the No. 1 virtual needs to be recoded into the host node, and the gene A is selected according to the constraint condition to complete the distribution;
4.3.5 if all genes do not meet the requirements, regenerating a new gene fragment according to the VHAM-R model. Interchanging two parent individuals, and repeating the intersection process to generate a second child individual;
4.4 mutation operation in the genetic algorithm is also obtained by simulating the evolution process of nature, the mutation is an important link in the genetic algorithm, and genes (including high-quality genes and poor-quality genes) which never appear in the parent individuals can be obtained through mutation with small probability, and the detailed description of the mutation operation is combined with the figure 4;
4.4.1 determining the individual chromosome gene to be mutated by the mutation function, as shown in the following formula.
Figure BDA0001801645280000181
Wherein U isj-pes、Uj-ramRespectively the CPU and the memory utilization rate of the host;
4.4.2 selection of fc(j) Deleting smaller genes to ensure that each time of deletion is poorer genes with lower utilization rate, wherein the variation function of the gene C in the figure 4 is smaller, and the gene C is removed;
4.4.3 the virtual machine No. 3 and the virtual machine No. 8 on the gene C in the figure 4 are coded again by a 4.3 cross operation method, the virtual machine No. 3 is inserted into the gene A, and the virtual machine No. 8 is inserted into the gene G.

Claims (2)

1. A virtual machine placement genetic optimization method based on a VHAM-R model in a cloud computing environment is characterized by comprising the following steps:
the first step is as follows: the following formalized description is proposed for the virtual machine placement problem, the process is as follows:
1.1 define the placement environment, and the data center has a physical host set PM ═ PM1，pm2，...，pmnN, the number of hosts is n, and a virtual machine set VM ═ VM needs to be placed1，vm2，...，vmmAnd where the number of virtual machines is m, and assuming that the number of virtual machines m is greater than or equal to the number of hosts n, defining a set of virtual machine placement groups P ═ { P ═ P1，p2，...，phH is the number of the placing groups;
1.2 defining resource states, vm for a given virtual machineiDefinition of
Figure FDA0003249468300000011
For virtual machines vmiThe required resources of the CPU are used,
Figure FDA0003249468300000012
for virtual machines vmiRequired memory resource, Vi-pesFor virtual machines vmiCPU utilization of Wi-ramFor virtual machines vmiMemory utilization of, for a given host pmjDefinition of
Figure FDA0003249468300000013
Is a main engine pmjThe current resources of the CPU are free from the CPU,
Figure FDA0003249468300000014
is a main engine pmjFree resources of memory, Uj-pesIs a main engine pmjCPU utilization, Uj-ramIs a main engine pmjThe memory utilization rate of (1) then defines the host pmjResource utilization rate UjComprises the following steps:
Uj＝αUj-pes+βUj-ram
alpha is more than 0 and less than 1, beta is more than 0 and less than 1, and alpha + p is 1;
definition of TagijAt the current time t, the host pmjSatisfying virtual machine vmiResource requirements of, i.e.
Figure FDA0003249468300000015
1.3 host availability, availability of a node means the probability of a node working at any time during the entire service time, for any network component i, its availability AiObtained by the following formula:
Figure FDA0003249468300000016
wherein MTTF represents mean time to failure, MTTR represents mean time to repair, assuming that the value of server availability is known and the availability between servers is independent and independent of each other;
1.4 calculating the power consumption, and in a cloud data center with n running physical hosts, for any physical host pmjE PM, the power consumption at a certain time t is shown by the following formula:
Figure FDA0003249468300000017
wherein c isjFor static energy consumption marking, fj(t) is time t host pmjCPU frequency, CPU utilization of Uj-pes(t), k is a constant coefficient, i.e. the power consumption is to some extent based on a linear model of the CPU utilization,
1.5 defining virtual machine Placement, VM set by Placement group pkE.g. P, selecting a host in the corresponding physical host set to complete placement mapping, and defining a virtual machine placement matrix M by meeting various constraint conditions in the placement processk[i][j]If M is presentk[i][j]1 denotes the placement group pkPlace virtual machine j on physical host i, otherwise, if M isk[i][j]0 denotes the placement group pkIn, virtual machine j is not placed on physical host i;
the second step is that: setting constraint conditions and optimization targets for virtual machine placement, wherein the process is as follows:
2.1 regarding the virtual machine placement problem in the cloud environment, not only needs to consider meeting the requirements of virtual machine resources, but also needs to consider how to reduce the energy consumption of a data center and efficiently utilize the resources, and in addition, needs to consider the availability problem of the placement request; therefore, the constraints to be considered are: the maximum use number of the server nodes is minimum, the energy consumption is minimum, the load is balanced, and the availability of the placement request is high, and the following constraint conditions are provided:
2.1.1 placing constraints, arbitrary virtual machine vmiUnder the same placing group, the server node can be placed on only one server node;
the constraints represent:
for the
Figure FDA0003249468300000018
In which a group p is placedk∈P；
In the same placement group, a single virtual machine is considered to be deployed and operated on only one server node;
2.1.2 resource constraints, for any server nodeIn other words, the consumption of each resource type should not exceed an upper limit, defining a server pmjRespectively, of CPU and memory capacity
Figure FDA0003249468300000021
And
Figure FDA0003249468300000022
represents;
the constraints represent:
for the
Figure FDA0003249468300000023
Is provided with
Figure FDA0003249468300000024
The parameter r is a constant coefficient, a part of resources are required to be reserved by the server node to ensure the normal operation of the server node, and r is less than or equal to 1;
2.1.3 reachability constraints, defining a function F (m, n, D) for representing reachability of inter-node communications, for any link (m, n) e.L, if communication delays of points m and n are at most D, the function F (m, n, D) returns 1, otherwise returns 0;
2.2, the virtual machine placement problem has a plurality of optimization targets, and the virtual machine placement problem is optimized and researched in two aspects of availability and energy consumption;
2.2.1 usability optimization
Assuming that the user request consists of a virtual machine between n different VM pairs with associated communication requirements, it is placed on the same server node pmjNot more than once, the usability of the placement cannot be improved because when pmjAt failure, all placement is at pmjWill fail simultaneously, and thus it is desirable to try to get vm toiPlaced on different nodes to increase availability; by HiTo represent placement of virtual machines vmiThe maximum number of nodes, i.e. HiRepresents vmiThe maximum number of server nodes that can be placed; definition of
Figure FDA0003249468300000025
The node number of the n virtual machines is H at most;
the availability definition and calculation of virtual machine placement can be divided into three categories: single placement, fully protected placement, and partially protected placement;
2.2.1.1 Single Placement
Single placement means that each virtual machine is placed on only one server node, i.e. H ═ 1, in the case of single placement, if the availability of n server nodes is a respectively1，A2，...，AnK virtual machines (n ≦ k) are placed on the n nodes, then the availability of the virtual machine placement scheme may be used as ApExpressed, defined as follows:
Figure FDA0003249468300000026
since the request contains k virtual machines, the probability that all the k virtual machines are running needs to be considered when calculating the availability;
2.2.1.2 complete protective Placement
Full protection placement refers to placement of arbitrary virtual machines
Figure FDA0003249468300000027
Are all placed in group pi(i is more than or equal to 1 and less than or equal to H) are placed on H different nodes; therefore, it can be considered that a full protection placement solution P may be composed of H single placement solutions, and within each single placement solution, placement, resource, and communication reachability constraints should be satisfied between pairs of virtual machines;
availability of a full protection placement solution is the probability that there is at least one placement group working within the life cycle of the service, and the availability is calculated as follows:
Figure FDA0003249468300000031
2.2.1.3 partial protection Placement
Partial protection placement means presence of virtual machine vmiE.g. VM, placed on less than H different nodes, i.e. two or more placement groups will virtual machine VMiPlaced on the same node and having a certain virtual machine vmjE.g. VM, so that H is greater than 1, and under the condition of placement of partial protection, if a virtual machine is placed on less than H nodes, the virtual machine can be considered to be placed by a plurality of placement groups together; the availability of the server node cannot be directly calculated by the formula in 2.2.1.2 because the availability of the server node where the shared virtual machine is placed is calculated twice; to handle this type of placement, redefine the operator-; suppose there are n nodes pm1，pm2，...，pmnTheir availability is A1，A2，...，An(ii) a For availability of AxNode pmxThe following definition is given for the operator · s:
Figure FDA0003249468300000032
then according to the formula in the above, define
Figure FDA0003249468300000037
For the operation between different sets, the availability of protection placement is calculated by the following formula:
Figure FDA0003249468300000033
2.2.2 energy consumption optimization
According to the formula in 1.4, during the period T, the physical host pmjTotal energy consumption of
Figure FDA0003249468300000034
Expressed as:
Figure FDA0003249468300000035
therefore, the following formula shows that the total energy consumption E of the servers of the data center is within the T periodTFor the sum of the energy consumptions of the various running servers:
Figure FDA0003249468300000036
the third step: creating a model
Based on the constraint conditions and the optimization target of virtual machine placement given in the second step, a virtual hierarchical structure model VHAM-R based on a Rendervous hash algorithm is established for optimizing and deciding the selection process of the virtual machine to the host, and the steps are as follows:
3.1 initializing a host set PM and a virtual machine set VM, wherein the number of the virtual machines to be placed in the group H at most and the availability set A of host nodes are obtained, if the number n of the host is less than 4, the step 3.2 is carried out, otherwise, the step 3.3 is carried out;
3.2 when the number n of hosts is less than 4, i.e. the number 2 x 2 of the minimum bi-level virtual structure cannot be constructed, defining that the host has a set W of assigned weight scores corresponding to each hosti＝{wi1，wi2，...，wikWhere k.gtoreq.n defines wijFor virtual machines vmiAt host pmjA weight score of (1), wij＝h(vmi，pmj) Wherein the function h (vm, pm) is a hash function agreed in the Renderkvous hash algorithm, and then passes through h (vmi，pmj) Virtual machine vmiIs assigned to a weight wijMaximum host pmjIf the host pmkIs h times that of other hosts, pmkThe equal share is divided into h shares, and the probability of the virtual machine distributed to the host is h times of that of other hosts;
3.3 the number n of the hosts is more than or equal to 4, the cluster of the hosts is divided, firstly, a constant z is selected, namely the number of the hosts in each cluster is z, and the set of the hosts is c-ceiling (n/z), wherein the ceiling function represents dividing n by zRounding up the value to the nearest integer, C0＝{cpm1，cpm2，...，cpmz}，C1＝{cpmz+1，cpmz+2，...，cpm2zUntil each host belongs to a cluster, each cluster is the lowest node in the virtual hierarchical structure;
the fourth step: the genetic algorithm operation improvement based on the VHAM-R model comprises the following steps:
4.1 for solving the process of placing the virtual machine in the server node, a grouping coding mode based on the host node cluster is provided, P represents an individual for the placing group, CiRepresents a host cluster CiCorresponding to chromosomes, and corresponding to genes on the host computer on each host computer cluster;
4.2, selecting individuals containing good genes in the population through a fitness function set by the selection operation, and taking the energy consumption of the data center and the placement availability of the virtual machines as optimization targets; the process of the selection operation is as follows:
4.2.1 when H1, for each individual, there is a virtual machine placement case, so its availability is calculated as a single placement, i.e. for individual P, its placement availability
Figure FDA0003249468300000041
Wherein the individual comprises n genes;
4.2.2 when H >1, availability for that H individual needs to be handled in a computationally intensive manner for protected placement;
4.2.2.1 for arbitrary virtual machines vmiE.g. VM, each of which is placed a group piThe virtual machines are placed on H different nodes, i is more than or equal to 1 and less than or equal to H, namely for H individuals, all the virtual machines on each gene are different, and under the condition, the usability of the H placement groups is processed according to a calculation mode of complete protection placement;
4.2.2.2 virtual machine vm if presentiE VM, virtual machine VM in two or more individualsiPlaced on the same gene, in which case the availability of the H placement groups is handled in a computational manner of partial protection placement;
4.2.3 combine data center energy consumption optimization to give the fitness function of the following genetic algorithm:
Figure FDA0003249468300000042
wherein E isminThe minimum value of the energy consumption of the data center in the T time period is obtained;
Figure FDA0003249468300000043
energy consumption for a single individual; when H ═ 1, availability is calculated in a single placement manner; when H >1, the availability calculations are divided into full protection placements and partial protection placements, where x is the number H of individual groups; for an individual or an individual group x, if the energy consumption of the data center is lower and the availability is higher, the value of the fitness function is larger, and the selected probability of the good genes being inherited to the next generation is higher;
4.3 crossover operation in genetic algorithm is to simulate the mating process between individuals in the biological world, and achieve gene recombination between parents through mating, so that the offspring obtains a new chromosome containing an excellent gene and generates more excellent offspring; the process of the crossover operation is as follows:
4.3.1 first choose two parents to be mated, named X, Y, randomly choose a certain node cluster containing one or more genes in the X parent as a part to be crossed, insert the node cluster, namely all the genes in the node cluster, into the cross point of the Y parent, and then generate new filial generation containing X, Y parent genes;
4.3.2 after completing gene insertion, because the invention uses the chromosome grouping coding mode based on the host cluster, the same host cluster may appear, if the situation appears, the inserted gene is merged into the original host cluster;
4.3.3 if the same two virtual machines exist on different host nodes, temporarily removing the host nodes which previously contain the same virtual machines from the chromosome codes;
4.3.4, the host nodes are temporarily eliminated, and virtual machine nodes which are not deployed by other hosts are possibly included, for the situation, the virtual machines need to be recoded into the host nodes, and genes which meet the constraint conditions and have the lowest energy consumption and the highest availability in the chromosome are selected to be distributed;
4.3.5 if all genes do not meet the requirements, regenerating a new gene segment according to the VHAM-R model, interchanging two parent individuals, and repeating the crossing process to generate a second filial generation individual;
4.4 mutation operation in genetic algorithm, the process of mutation operation is:
4.4.1 determining the individual chromosome gene to be mutated by the mutation function, as shown in the following formula:
Figure FDA0003249468300000051
wherein U isj-pes、Uj-ramRespectively the CPU and the internal memory utilization ratio of the host,
Figure FDA0003249468300000052
is a set parameter;
4.4.2 selection of fc(j) The smaller gene is deleted, so that the gene with lower utilization rate is deleted every time,
4.4.3 inserting the virtual machine on the gene into other genes by a 4.3 cross operation method.
2. The method for genetic optimization of placement of virtual machines based on VHAM-R model in cloud computing environment according to claim 1, wherein in said step 3.3, the process of clustering the hosts is:
3.3.1 determining virtual leaf node sectors and the depth of a virtual hierarchical structure, selecting the leaf number f of each sub-node sector in the virtual hierarchical structure, wherein f is an integer, and selecting proper f and z can enable the obtained algorithm benefit and load balance degree to be approximate to the expectation; according to the leaf number f of the node sector and the number c of the host clusters, the depth d of the virtual hierarchical structure can be obtained:
fd≥c
wherein d is the smallest positive integer, and d is calculated so that the above formula holds;
3.3.2 numbering each virtual leaf node sector, wherein each sector is uniformly numbered by adopting a natural number from 0, 1, 2.
3.3.3 for a certain virtual machine vmiFor any virtual node s, there is a corresponding weight wis＝h(vmi，s)，h(vmiS) contains a hash function agreed upon, which passes through h (vm) at each level of the virtual hierarchyiS) calculating the weight of each virtual node, selecting the node with the highest score to continue layering downwards until the real host node cluster Cx to the bottom layer is selected;
3.3.4 virtual machine vmiSelecting a real host node cluster CxLater, when real node selection is carried out, the fact that any real node cluster C exists is assumedxHost node cpm of (1)xz+jAll have a corresponding weight score Wi(xz+j)＝H(vmi，cpmxz+j)*Tagi(xz+j)If Tag is false, the value is 0, and if true, the value is 1; wherein the virtual machine vm is usediCpm assigned to hostxz+jThen, H (vm)i，cpmxz+j) For the same period of T, EoldAnd assigning virtual machines vmiReal host node cluster CxRatio of total power consumption of to the host pmxz+jResource utilization rate Uxz+jProduct of the difference with 1 and the corresponding weight constant and sum of host availability and coefficient product:
Figure FDA0003249468300000053
Figure FDA0003249468300000061
wherein Exz+jFor the cpm of the host in the T time periodxz+jEnergy consumption of EoldMeans that when no new virtual machine is allocated in the same T time period, the real host node cluster CxEnergy consumption of (2); α, β, γ are weights representing the three; a. thexz+jCpm for the hostxz+jAvailability of (2);
3.3.5 loop through steps 3.3.3-3.3.4 until all virtual machines vmiFinally, the weighting score W is selectedi(xz+j)The highest host node completes the allocation.
CN201811079838.9A 2018-09-17 2018-09-17 Virtual machine placement genetic optimization method based on VHAM-R model in cloud computing environment Active CN109447264B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811079838.9A CN109447264B (en) 2018-09-17 2018-09-17 Virtual machine placement genetic optimization method based on VHAM-R model in cloud computing environment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811079838.9A CN109447264B (en) 2018-09-17 2018-09-17 Virtual machine placement genetic optimization method based on VHAM-R model in cloud computing environment

Publications (2)

Publication Number Publication Date
CN109447264A CN109447264A (en) 2019-03-08
CN109447264B true CN109447264B (en) 2021-11-23

Family

ID=65532534

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811079838.9A Active CN109447264B (en) 2018-09-17 2018-09-17 Virtual machine placement genetic optimization method based on VHAM-R model in cloud computing environment

Country Status (1)

Country Link
CN (1) CN109447264B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111124590A (en) * 2018-10-31 2020-05-08 千寻位置网络有限公司 Virtual machine placing method and device and server
CN110308993B (en) * 2019-06-27 2022-12-13 大连理工大学 Cloud computing resource allocation method based on improved genetic algorithm
JP2021149129A (en) * 2020-03-16 2021-09-27 富士通株式会社 Fee calculation program and method for calculating fee
CN112286677B (en) * 2020-08-11 2021-07-16 安阳师范学院 Resource-constrained edge cloud-oriented Internet of things application optimization deployment method
CN112732408A (en) * 2021-01-18 2021-04-30 浪潮云信息技术股份公司 Method for computing node resource optimization

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102984137A (en) * 2012-11-14 2013-03-20 江苏南开之星软件技术有限公司 Multi-target server scheduling method based on multi-target genetic algorithm
CN103024048A (en) * 2012-12-17 2013-04-03 南京邮电大学 Resources scheduling method under cloud environment
CN103576829A (en) * 2012-08-01 2014-02-12 复旦大学 Hybrid genetic algorithm based dynamic cloud-computing virtual machine scheduling method
US9317337B2 (en) * 2012-04-13 2016-04-19 International Business Machines Corporation Utilizing software component metadata to provision virtual machines in a networked computing environment
CN105740051A (en) * 2016-01-27 2016-07-06 北京工业大学 Cloud computing resource scheduling realization method based on improved genetic algorithm
CN107193640A (en) * 2017-06-26 2017-09-22 浙江工业大学 A kind of virtual machine initial placement strategy process based on Rendezvous hash algorithms
CN107329815A (en) * 2017-06-26 2017-11-07 浙江工业大学 A kind of cloud task load equalization scheduling method searched for based on BP Tabu
WO2017192343A1 (en) * 2016-05-06 2017-11-09 Microsoft Technology Licensing, Llc Cloud storage platform providing performance-based service level agreements
US9947236B2 (en) * 2012-07-10 2018-04-17 Inetwork, Inc. Apparatus, system, and method for a virtual instruction cloud
CN108009003A (en) * 2017-12-01 2018-05-08 湖南女子学院 A kind of multiple target virtual machine adaptive location system of selection, distributed cloud system

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9317337B2 (en) * 2012-04-13 2016-04-19 International Business Machines Corporation Utilizing software component metadata to provision virtual machines in a networked computing environment
US9947236B2 (en) * 2012-07-10 2018-04-17 Inetwork, Inc. Apparatus, system, and method for a virtual instruction cloud
CN103576829A (en) * 2012-08-01 2014-02-12 复旦大学 Hybrid genetic algorithm based dynamic cloud-computing virtual machine scheduling method
CN102984137A (en) * 2012-11-14 2013-03-20 江苏南开之星软件技术有限公司 Multi-target server scheduling method based on multi-target genetic algorithm
CN103024048A (en) * 2012-12-17 2013-04-03 南京邮电大学 Resources scheduling method under cloud environment
CN105740051A (en) * 2016-01-27 2016-07-06 北京工业大学 Cloud computing resource scheduling realization method based on improved genetic algorithm
WO2017192343A1 (en) * 2016-05-06 2017-11-09 Microsoft Technology Licensing, Llc Cloud storage platform providing performance-based service level agreements
CN107193640A (en) * 2017-06-26 2017-09-22 浙江工业大学 A kind of virtual machine initial placement strategy process based on Rendezvous hash algorithms
CN107329815A (en) * 2017-06-26 2017-11-07 浙江工业大学 A kind of cloud task load equalization scheduling method searched for based on BP Tabu
CN108009003A (en) * 2017-12-01 2018-05-08 湖南女子学院 A kind of multiple target virtual machine adaptive location system of selection, distributed cloud system

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
Auto-tuning Performance of MPI Parallel Programs Using Resource Management in Container-Based Virtual Cloud;Hongyi Ma,et al.;《2016 IEEE 9th International Conference on Cloud Computing (CLOUD)》;20170119;全文 *
Data-Aware Virtual Machine Placement and Rate Allocation in Cloud Environment;Kamran Zamanifar, et al.;《2012 Second International Conference on Advanced Computing & Communication Technologies》;20120315;全文 *
Virtual Machine Placement in Predictable Computing Clouds;Richard Rauscher,et al.;《2014 IEEE 7th International Conference on Cloud Computing》;20141204;全文 *
面向多源异构数据的云推送平台的研究与应用;陆佳炜等.;《计算机科学》;20160630;全文 *

Also Published As

Publication number Publication date
CN109447264A (en) 2019-03-08

Similar Documents

Publication Publication Date Title
CN109447264B (en) Virtual machine placement genetic optimization method based on VHAM-R model in cloud computing environment
Iranmanesh et al. DCHG-TS: a deadline-constrained and cost-effective hybrid genetic algorithm for scientific workflow scheduling in cloud computing
Alresheedi et al. Improved multiobjective salp swarm optimization for virtual machine placement in cloud computing
Szabo et al. Science in the cloud: Allocation and execution of data-intensive scientific workflows
CN106227599B (en) The method and system of scheduling of resource in a kind of cloud computing system
Zhao et al. An energy-aware algorithm for virtual machine placement in cloud computing
CN106506657A (en) One kind distributes method of adjustment based on multiobject cloud computing virtual machine
CN108108224B (en) Virtual machine placement method in cloud data center based on ant colony optimization algorithm
CN108469983A (en) A kind of virtual machine deployment method based on particle cluster algorithm under cloud environment
CN106484512B (en) The dispatching method of computing unit
CN108170530B (en) Hadoop load balancing task scheduling method based on mixed element heuristic algorithm
CN108009003A (en) A kind of multiple target virtual machine adaptive location system of selection, distributed cloud system
CN105843666B (en) Based on the virtual machine placement method of multiple-objection optimization in cloud computing
CN112286677A (en) Resource-constrained edge cloud-oriented Internet of things application optimization deployment method
CN105550033A (en) Genetic-tabu hybrid algorithm based resource scheduling policy method in private cloud environment
Idrissi et al. A new approach for a better load balancing and a better distribution of resources in cloud computing
CN110008023B (en) Cloud computing system budget constraint random task scheduling method based on genetic algorithm
Supreeth et al. Hybrid genetic algorithm and modified-particle swarm optimization algorithm (GA-MPSO) for predicting scheduling virtual machines in educational cloud platforms
Yousefipour et al. Improving the load balancing and dynamic placement of virtual machines in cloud computing using particle swarm optimization algorithm
Lu et al. Optimal machine placement based on improved genetic algorithm in cloud computing
CN111176784A (en) Virtual machine integration method based on extreme learning machine and ant colony system
CN104580518A (en) Load balance control method used for storage system
Dhib et al. Cost-aware virtual machines placement problem under constraints over a distributed cloud infrastructure
CN112187535B (en) Server deployment method and device in fog computing environment
CN109343933B (en) Virtual machine initial placement strategy method based on improved genetic algorithm

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant