CN111309472A - Online virtual resource allocation method based on virtual machine pre-deployment - Google Patents
Online virtual resource allocation method based on virtual machine pre-deployment Download PDFInfo
- Publication number
- CN111309472A CN111309472A CN201910625142.XA CN201910625142A CN111309472A CN 111309472 A CN111309472 A CN 111309472A CN 201910625142 A CN201910625142 A CN 201910625142A CN 111309472 A CN111309472 A CN 111309472A
- Authority
- CN
- China
- Prior art keywords
- virtual machine
- deployment
- virtual
- resource allocation
- online
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 95
- 238000005457 optimization Methods 0.000 claims abstract description 69
- 238000013468 resource allocation Methods 0.000 claims abstract description 46
- 230000004044 response Effects 0.000 claims abstract description 6
- 230000008569 process Effects 0.000 claims description 21
- 230000003068 static effect Effects 0.000 claims description 21
- 238000004364 calculation method Methods 0.000 claims description 14
- 239000013598 vector Substances 0.000 claims description 13
- 238000013178 mathematical model Methods 0.000 claims description 11
- 230000000694 effects Effects 0.000 claims description 10
- 238000007619 statistical method Methods 0.000 claims description 10
- 238000007726 management method Methods 0.000 abstract description 3
- 239000011159 matrix material Substances 0.000 description 11
- 230000006870 function Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 4
- 230000007704 transition Effects 0.000 description 4
- 230000008859 change Effects 0.000 description 2
- 230000003247 decreasing effect Effects 0.000 description 2
- 238000010998 test method Methods 0.000 description 2
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 description 1
- 238000007476 Maximum Likelihood Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000004576 sand Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5077—Logical partitioning of resources; Management or configuration of virtualized resources
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention relates to an online virtual resource allocation method based on virtual machine pre-deployment, belonging to the field of cloud computing and data center resource management; an online virtual resource allocation framework based on virtual machine pre-deployment is constructed, and the framework realizes the organic combination of virtual machine deployment offline optimization and virtual machine deployment online decision through a virtual machine pre-deployment mode; secondly, designing an online virtual resource allocation flow based on the framework to realize rapid virtual resource allocation, and simultaneously, completing resource allocation by preferentially using a pre-deployed virtual machine to improve the utilization rate of physical resources of the data center after resource allocation; the method and the device can realize quick response to the virtual machine request through the online deployment of the virtual machine, and can perform offline resource optimization on the deployment of the virtual machine, so that the utilization rate of physical resources of the cloud data center is improved.
Description
Technical Field
The invention relates to an online virtual resource allocation method based on virtual machine pre-deployment, and belongs to the field of cloud computing and data center resource management.
Background
In a cloud environment, the computing resources of a data center are typically virtualized into different types of virtual machines and provided to users to perform computing tasks with different resource requirements. Different types of virtual machines have different resource configurations (e.g., "m 3. large" type virtual machine provided by AmazonEC2 is configured with 2 CPUs and 7.5G memory capacity). The user can submit the resource allocation request according to the specific resource requirement of the computing task to be executed so as to obtain the virtual machine instance of the specified type, and further the execution of the computing task in the virtual machine instance is realized. When a user submits a resource allocation request, a virtual machine instance of a corresponding type needs to be actually deployed in a physical machine and provided to the user in response to the resource allocation request of the user, which is called virtual resource allocation.
In the current cloud environment, virtual resource allocation needs to implement online virtual machine deployment to quickly respond to a virtual resource allocation request submitted by a user, so that it is ensured that a virtual machine can be rapidly and timely deployed and provided to the user, and thus user experience when the user makes a resource request is ensured. In addition, the current cloud data center faces large-scale virtual resource allocation requests, and hundreds of heterogeneous virtual machine instances are generally deployed at the same time; therefore, when virtual resource allocation is performed, the virtual machine corresponding to the large-scale heterogeneous virtual machine allocation request needs to be reasonably deployed in the physical machine, so as to improve the resource utilization rate of the physical machine.
The related art currently performing virtual resource allocation is generally classified into an online method and an offline method. Although online methods in the related art can implement online virtual machine deployment, these methods cannot optimize virtual machine deployment based on offline information, which results in a problem of low system resource utilization after resource allocation. Another part of resource allocation techniques focusing on offline methods generally only consider static virtual machine deployment issues, which cannot respond to online-generated virtual machine allocation requests in time; in addition, these methods usually model the virtual machine deployment problem into a boxing problem for solution, and when the large-scale heterogeneous virtual machine deployment problem is faced, this modeling approach can lead to a problem that is too complex and difficult to solve. Therefore, the main problem to be solved by the invention is how to perform online virtual resource allocation aiming at a virtual resource allocation request dynamically submitted by a user, and the invention improves the utilization rate of physical resources of a data center while rapidly responding to the allocation request of a large-scale heterogeneous virtual machine so as to realize resource allocation optimization.
Disclosure of Invention
The invention aims to provide an online virtual resource allocation method based on virtual machine pre-deployment, which realizes the organic combination of the offline optimization of virtual machine deployment and the online decision of virtual machine deployment in a virtual machine pre-deployment mode, can realize the quick response to the virtual machine request through the virtual machine online deployment, and can also perform the offline resource optimization on the virtual machine deployment, thereby improving the utilization rate of physical resources of a cloud data center.
In order to achieve the purpose, the invention adopts the technical scheme that: the allocation method comprises the steps that firstly, an online virtual resource allocation framework based on virtual machine pre-deployment is constructed, and the framework realizes the organic combination of virtual machine deployment offline optimization and virtual machine deployment online decision through a virtual machine pre-deployment mode; secondly, designing an online virtual resource allocation flow based on the framework to realize rapid virtual resource allocation, and simultaneously, completing resource allocation by preferentially using a pre-deployed virtual machine to improve the utilization rate of physical resources of the data center after resource allocation; then, based on the framework and the process, a method for determining the optimal virtual machine pre-deployment scale under the resource allocation framework based on a queuing theory model is designed, and the offline resource allocation optimization effect is improved by reasonably determining the scale of offline optimization; finally, a mathematical model of the virtual machine pre-deployment scheme optimization problem in the offline optimization is constructed, the solving complexity of the virtual machine pre-deployment optimization problem in a large-scale environment is effectively reduced, the problem is solved, and finally the physical resource utilization rate is improved through reasonable deployment of heterogeneous virtual machines in the physical machine;
the method specifically comprises the following steps:
step 2, designing an online virtual resource allocation process based on the frame constructed in the step 1;
and 4, determining an optimal virtual machine deployment scheme according to the virtual machine pre-deployment scale determined in the step 3 so as to improve the physical resource utilization rate after the virtual machine deployment.
Preferably, the online virtual resource allocation framework in step 1 is divided into a static part and a dynamic part; in the static part, the deployment of the virtual machine in the physical machine is optimized in an off-line manner through the static pre-deployment of the virtual machine, so that the resource utilization rate of the physical machine is improved; in the dynamic part, the online dynamic deployment of the virtual machine is carried out according to the dynamic arrival of the resource allocation request, so that the quick response to the resource allocation request is realized; on one hand, the online virtual resource allocation framework realizes online resource allocation of dynamic arrival resource requests; and on the other hand, the offline optimization of resource allocation is realized through the optimization of the pre-deployment virtual machine.
Preferably, the online virtual resource allocation process in step 2 includes the following steps:
step 201, determining the type of a virtual machine required by responding to a current arriving virtual resource allocation request;
step 202, aiming at the type of the virtual machine corresponding to the resource allocation request, if the distributable deployed type virtual machine exists in the system, distributing the corresponding virtual machine to the resource allocation request, thereby completing the virtual resource allocation;
step 203, if the system does not have the distributable deployed virtual machine of the type currently, dynamically deploying one virtual machine of the type, and distributing the virtual machine for the current resource distribution request, thereby completing the virtual resource distribution.
Preferably, the method for determining the optimal virtual machine pre-deployment scale in step 3 includes the following steps:
301, carrying out statistical analysis on the dynamic arrival condition of the resource allocation request, and modeling the deployment demand of each type of virtual machine on the basis of the statistical analysis;
step 302, constructing a pre-deployment scale optimization model based on a queuing theory, and determining a functional relation between the scale of the pre-deployment virtual machine and the utilization rate of the pre-deployment virtual machine;
and 303, determining the optimal virtual machine pre-deployment scale by using a binary search method according to the functional relation determined in the step 302.
Preferably, the step 4 comprises the following steps:
step 401, determining the number of each type of virtual machine to be pre-deployed according to the optimal virtual machine pre-deployment scale determined in step 303 of claim 4;
step 402, constructing a mathematical model of a virtual machine pre-deployment scheme optimization problem;
and 403, solving the mathematical model of the optimization problem in the step 402 according to the number of the virtual machines of each type to be pre-deployed, which is determined in the step 401, and determining a final virtual machine deployment scheme.
Compared with the prior art, the online virtual resource allocation method based on virtual machine pre-deployment provided by the invention has the following advantages:
1. the method realizes the organic combination of the offline optimization of the deployment of the virtual machines and the online decision of the deployment of the virtual machines in a virtual machine pre-deployment mode, can realize the quick response to the request of the virtual machines through the online deployment of the virtual machines, and can also perform the offline resource optimization on the deployment of the virtual machines, thereby improving the resource utilization rate of the system.
2. The concept of "virtual machine deployment mode" is introduced in virtual resource allocation based on the virtual machine type. Problem modeling is carried out on the large-scale virtual machine deployment optimization problem on the basis of the virtual machine deployment mode, so that the solving scale of the optimization problem can be effectively reduced, and the solving efficiency of the optimization problem is improved.
3. The simplex method based on the delay column generation method is adopted to solve the deployment optimization problem of the virtual machines, the solving process is simple and effective, the optimization effect is good, and the method can be applied to large-scale data center environments.
4. The online method and the offline method are organically combined in online virtual resource allocation, and a new idea is provided for effectively solving the problem of dynamic resource management of the cloud computing data center.
Description of the figures
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts
Fig. 1 shows the optimization effect of resource allocation for different virtual machine deployment schemes.
Fig. 2 shows the dynamic and static part division of the deployment load of the virtual machines in the data center (dynamic area above the dotted line and static area below the dotted line) (time (hours) in the horizontal direction; number of virtual machines in the data center in the vertical direction).
FIG. 3 is a diagram of an online virtual resource allocation framework based on virtual machine pre-deployment.
FIG. 4 is a flow diagram of online virtual resource allocation based on virtual machine pre-deployment.
FIG. 5 is an M/M/n/n queuing model for data center s-type computation task arrival and execution.
FIG. 6 is a state transition diagram of a data center s-type compute task M/M/n/n queuing system.
FIG. 7 is a relationship between virtual machine usage efficiency and the number of pre-deployed virtual machines.
Detailed Description
The technical solution of the present invention will be described in detail below with reference to the accompanying drawings.
Referring to fig. 1-7, virtual resource allocation in a cloud data center essentially refers to a process of completing actual deployment of virtual machines on physical machines according to virtual machine allocation requests. And whether the deployment of the virtual machine in the physical machine reasonably and directly influences the optimization of the utilization rate of the system resources. As shown in fig. 1, for a virtual machine allocation request of three heterogeneous virtual machine types, different deployment schemes may result in different resource optimization effects, and compared to the deployment scheme B, the deployment scheme a uses fewer physical machines, and has a higher system resource utilization rate and a better resource optimization effect. Current virtual resource allocation technical solutions to implement online virtual resource allocation, an online virtual machine deployment method is generally adopted. Although the method can realize rapid virtual machine deployment, the resource utilization rate of the system after virtual machine allocation cannot be effectively improved due to the lack of offline optimization. In contrast, although the offline virtual machine deployment method can perform offline optimization on the deployment scheme, the offline virtual machine deployment method can perform overall optimization on virtual machine deployment only according to static information regularly, and cannot achieve online virtual machine rapid deployment.
Therefore, the core idea of the online virtual resource allocation method provided by the invention is as follows: the online calculation method and the offline method are organically combined, so that the online method is utilized to realize the rapid deployment of the virtual machine, and the offline method is utilized to perform the offline optimization of the deployment of the virtual machine.
Fig. 2 shows a change of the number of a certain type of virtual machines deployed in the southeast university cloud data center within 100 hours. It can be seen from the figure that the shaded area representing the change of the number of virtual machines changes dynamically, but the lower half of the area is basically stable and unchanged. As shown in the figure, the area can be divided into an upper part and a lower part by drawing a straight line, which are a dynamic area and a static area respectively. The number of the virtual machines in the static region can be regarded as stable and unchanged, so that offline deployment can be performed in advance for the part of the virtual machines, and the deployment of the virtual machines in the physical machines is optimized through offline optimization, so that the resource utilization rate of the physical machines after deployment is improved. For the virtual machines in the dynamic region, an online deployment method can be adopted to deploy the virtual machines according to the dynamic arrival of the virtual machine allocation requests. Compared with the prior art that the whole area is deployed on line, the method for combining online deployment and offline deployment provided by the invention can effectively improve the resource utilization rate of the whole system through offline optimization of the virtual machine deployment in the static area.
Specifically, the online virtual resource allocation method based on virtual machine pre-deployment of the invention comprises the following steps:
step 2, designing an online virtual resource allocation process based on the frame constructed in the step 1;
and 4, determining an optimal virtual machine deployment scheme according to the virtual machine pre-deployment scale determined in the step 3 so as to improve the physical resource utilization rate after the virtual machine deployment.
The specific description of each step is as follows:
in step 1, based on the concept of organically combining the online computing method and the offline method, the invention designs an online virtual resource allocation framework based on virtual machine pre-deployment, which is shown in fig. 3 and divides the virtual machine deployment into a static part and a dynamic part. In the static part, the virtual machines are pre-deployed, so that the system can perform offline optimization on the deployment of the virtual machines in the physical machine, thereby improving the resource utilization rate of the physical machine. In the dynamic part, the system performs virtual machine deployment and completes virtual machine allocation when a resource request arrives. With the dynamic arrival of resource requests, the system needs to perform online virtual machine deployment. Therefore, on one hand, the online resource allocation framework realizes online resource allocation of the dynamically arrived resource request; and on the other hand, the offline optimization of resource allocation is realized through the optimization of the pre-deployment virtual machine.
In step 2), based on the above online virtual resource allocation framework, the present invention designs an online virtual resource allocation process as shown in fig. 4, when a resource allocation request arrives, first determining the type of virtual machine required by the resource allocation request, then trying to allocate a pre-deployed virtual machine to the resource allocation request, and if a deployed type of virtual machine exists in the system, allocating a corresponding virtual machine to the system, thereby completing resource allocation; if the system does not have the deployed virtual machine of the type currently, dynamically deploying one virtual machine of the type, and allocating the virtual machine for the resource request, thereby completing resource allocation. Because the static part of the pre-deployed virtual machines are optimized by offline deployment, the deployed physical machines have higher resource utilization rate, so the process can improve the system resource utilization rate after resource allocation by preferentially using the pre-deployed virtual machines to complete resource allocation.
The method for determining the optimal virtual machine pre-deployment scale in step 3 needs to introduce the following relevant variables and definitions of the optimization problem:
definition 1 (virtual machine type): the virtual machines are classified according to the amount of computing resources provided by the virtual machines, the virtual machines providing the same amount of computing resources are classified into the same type of virtual machines, and if the data center provides V types of virtual machines, a certain type of virtual machine can be represented as V e (1, …, V).
Definition 2 (calculation task type): the invention assumes that the computing tasks required to be executed by users of the data center are divided into S types in total, a certain computing task type can be expressed as S e (1, …, S), and the type of the virtual machine required by the type of the computing task is defined as vs∈(1,…,V)。
Definition 3 (calculate task arrival intensity): the statistical value of the number of the calculation tasks of a certain type submitted by the user in unit time is called the arrival intensity of the calculation tasks (called the arrival intensity for short), and the arrival intensity of the s-type calculation tasks is recorded as lambda in the inventions。
Definition 4 (calculating task execution rate): the number of s-type computation tasks completed in a virtual machine in a unit time, referred to as the execution rate of the computation tasks, is denoted as musThe execution time of the corresponding computing task in the virtual machine can be recorded as
Definition 5 (static pre-deployment virtual machine size): v required for s-type computation taskssThe method comprises the following steps that type virtual machines are statically allocated in a data center in advance, a corresponding number of the type virtual machines are allocated in the physical machines in advance, the number is called as the scale of the static pre-allocation virtual machines, and the pre-allocation number is defined as qs。
Definition 6 (online deployment virtual machine size): when a user submits an s-type computing task, if no idle pre-deployment virtual machine can be allocated to the task, the corresponding type of virtual machine needs to be deployed online, and the type of the virtual machine in the system is pointed outThe number of the virtual machines deployed on line for the s-type computing task is defined as the scale of the virtual machines deployed on line, and since the number of the virtual machines deployed on line changes dynamically, the average value of the number is considered and defined as ds。
Definition 7 (virtual machine usage): virtual machine usage refers to the ratio of virtual machines used in deployed virtual machines. For the online deployed virtual machines, since the virtual machines are dynamically deployed on demand, the number of virtual machines in use in the online deployed virtual machines is the deployment number d thereofs(ii) a For static pre-deployed virtual machines, the chapter defines nbusyRepresenting the number of virtual machines being used in the static pre-deployed virtual machines. The utilization of the virtual machine to which the s-type computing task belongs can be defined as
Define 8 (virtual machine usage optimization constraints): the resource allocation method provided by the invention hopes that the utilization rate of the virtual machine deployed for the type s computing task in the optimized system meets the following constraint:
v representing all deployments in a systemsA usage constraint value for the type virtual machine.
Optimizing the virtual machine pre-deployment scale: arrival intensity λ for s-type computing tasks in a given data centersExecution rate μsAnd type v of virtual machine requiredsOn the premise of meeting the optimization constraint of the utilization rate of the virtual machine, the scale of the virtual machine for offline pre-deployment of various types of computing tasks in the system is maximized, so that the offline resource optimization effect is improved. The problem can be formally defined as follows. It can be seen that the optimization problem contains only qs，nbusyAnd dsSeveral parameters, so the invention requires a function of the input parameter λs，μsConstruction of qs，nbusyAnd dsThe relationship between several parameters to solve the optimization problem described below.
max qs(1)
The method for determining the optimal virtual machine pre-deployment scale in the step 3 comprises the following steps:
step 301, performing statistical analysis on the dynamic arrival condition of the resource allocation request, and modeling the deployment demand of each type of virtual machine based on the statistical analysis:
in the cloud data center, the deployment demand of the virtual machines depends on the number of computing tasks with the deployment demand of the virtual machines of the type submitted by users, so the method carries out statistical analysis on the number of computing tasks of each type operated by the data center, and models are carried out on the demand of the virtual machines of each type. Firstly, based on the cloud data center of southeast university, the invention carries out statistical analysis on the arrival intensity and the task execution duration of each type of calculation task in the data center. In actual statistics, the statistical results are basically similar to those of other types of tasks, the actual statistical value of the task arrival intensity is found to be very similar to a corresponding poisson distribution theoretical value curve through the following statistical analysis, and the actual statistical value of the task execution duration is very similar to a corresponding negative exponential distribution theoretical value curve. Therefore, assuming that the arrival intensity of one type of task conforms to the poisson distribution, the execution duration distribution of the task conforms to the negative exponential distribution.
Next, the above two hypotheses were examined by the kolmogorov test method, and the significance level a was made 0.01. Taking the assumption that the task arrival intensity conforms to the poisson distribution as an example, firstly, the arrival intensity conforms to the poisson distribution, and a maximum likelihood estimation method is used for estimating a parameter lambda of the poisson distribution, so that a probability density function f (x) of the corresponding poisson distribution is obtained. Then, using a Kolmogorov test method to test whether the statistical value accords with the Poisson distribution, firstly, 200 pieces of arrival intensity statistical records are recorded according to the order from small to largeSorting, in turn, respectively denoted as x1,x2,…x200Respectively calculate
Then D can be calculatedn＝max{d1,d2,…,d2000.0357. For a given significance level a of 0.01, D is obtained by looking up the table200,0.010.1152 because 0.0357<0.1152, not in the reject domain, so the hypothesis is accepted. That is, the arrival intensity of the same type of calculation task in the data center is considered to be in accordance with the poisson distribution. Based on the above statistical analysis results, the following two inferences are given for modeling the deployment demand of each type of virtual machine.
Inference 1: the arrival process of a certain s-type calculation task is a Poisson process, the distribution of the number of task arrivals in each hour conforms to the Poisson distribution, and the probability that k task arrivals occur in each hour can be expressed as
Wherein λsThe average arrival intensity per hour for the tasks was calculated for the s type.
Inference 2: the execution time length distribution obeying parameter of a certain s type calculation task is
A negative exponential distribution of (1), wherein the probability that the task execution duration is t can be expressed as
Wherein musThe execution rate of the task is calculated for the s type.
Step 302, constructing a pre-deployment scale optimization model based on a queuing theory, and determining a functional relation between the scale of the pre-deployment virtual machine and the utilization rate of the pre-deployment virtual machine:
the arrival and execution process of a certain type of s-computing task in a data center can be modeled as a basic M/n/n queuing model, according to the results of two inferences modeling for deployment demand of each type of virtual machine. The invention is therefore directed to an s-type meterThe calculation task constructs a pre-deployment scale optimization model based on the queuing theory, so that the number q of pre-deployed virtual machines is maximized on the premise of meeting the optimization constraint of the utilization rate of the virtual machiness. The M/M/n/n queuing model for s-type computation task arrival and execution is shown in FIG. 5: the arrival process of the calculation task is conformed to average the intensity lambdasA poisson process as a parameter; the execution time of the task in the virtual machine is consistent with the average execution time
Is a negative exponential distribution of the parameter; when a task arrives, if an idle pre-deployment virtual machine exists currently, a scheduler schedules the task to the virtual machine for execution, after a certain time period of execution, the task is completed and leaves the system, and the current pre-deployment virtual machine becomes an idle state; if no idle pre-deployment virtual machine exists currently, the scheduler dynamically deploys a corresponding virtual machine online and schedules the task to the virtual machine for execution, and after the task is completed and executed, if no subsequent task continues to use the current virtual machine, the current virtual machine is released. Wherein P in the figurebAnd representing the probability that no idle pre-deployed virtual machine in the system is available for task execution, which is called system loss rate, and n represents the number of virtual machines currently pre-deployed in the system.
The M/M/n/n queuing model described above for the s-type computing task is actually a random process with a state {0,1,2 …, n }, where each state value represents the number of virtual machines being used in the n pre-deployed virtual machines in the queuing system. When the system is in state 1, namely one of the pre-deployed virtual machines is being used for executing the task, once the task is executed, the system state is in state 0, namely the transition intensity from state 0 → 1 is musWhen the system is in a state k (2 ≦ k ≦ n), namely, some k pre-deployed virtual machines are being used, once one of the virtual machines completes the task execution, the system is in a state k-1, and the transition intensity from the state k to the state k-1 is k μs. It can be seen that the process is a life-death process, the life-death rate and the death rate are shown in formula (3), and the state of the process is changedThe figure is shown in fig. 6. Let pikIf the system is in the state K, the K's equation under the equilibrium condition shown in equations (4) to (6) can be obtained according to the state transition diagram, and the probability that the system is in each state can be obtained according to the K's equation shown in equations (7) and (8).
For this system, when the system state is n, it means that all the pre-deployed virtual machines are in use, and therefore when a new computing task arrives, it is necessary to deploy the virtual machines online to complete the execution of the task, so the system loss rate P shown in fig. 5 isbThe number of virtual machines to be deployed online per unit time in the system is shown in formula (10), and the number of pre-deployed virtual machines to be deployed online per unit time in the system is shown in formula (11). The size of a static pre-deployed virtual machine in the current system is assumed to be n-qsThen the current system virtual machine utilization rate can be represented by U (q) shown in formula (12)s) And (4) calculating a function.
Step 303, determining an optimal virtual machine pre-deployment scale by using a binary search method according to the functional relationship determined in step 302 and shown in formula (12):
the function shown in equation (12) is qsAs a decreasing function of the variable, the invention
For example, the function is plotted as shown in FIG. 7, and the usage rate u of the system virtual machine can be seensWith pre-deployment of virtual machine scale qsIs increased and decreased. Therefore, the invention adopts a binary search method to calculate U (q)s) The value of the function is found so that
Maximum q ofsThe value is obtained. Specifically, an initial starting value x is first takenstartSo that
After selecting an initial end value xendSo that
Then calculate
And determining the trial calculation interval according to the value
Whether in the interval
Wherein the search is continued.
In step 4, the method for determining the optimal virtual machine deployment scheme according to the virtual machine pre-deployment scale determined in step 3 comprises the following steps:
step 401, determining the number of virtual machines of each type to be pre-deployed according to the optimal virtual machine pre-deployment scale determined in step 303: for a virtual machine type v, the number of virtual machines which need to be pre-deployed is defined as nv. The number q of pre-deployed virtual machines required for each type of computing task is determined by step 303sThen, the pre-deployment number b of the v-type virtual machinevCan be obtained by the following equation (13).
Step 402, constructing a mathematical model of the virtual machine pre-deployment scheme optimization problem:
the invention gives the following relevant definitions and specific problem modeling:
based on the description of the deployment mode (definition 1) of the virtual machine, a vector A corresponding to a feasible deployment mode ii＝[ai1,…,aiv,…,aiV]A constraint, as described in equation (14), must be satisfied that indicates that the resource requirements of a virtual machine deployed on a physical machine in a given mode cannot exceed the total capacity of resources that the physical machine can provide. Wherein r isvAnd C represents the resource capacity of the physical machine.
Based on the above definition, let n be the number of all possible virtual machine deployment modes, let xiAccording to a deployment mode Ai＝[ai1,…,aiv,…,aiV]Arranged byThe number of physical machines:
the optimization goal in the above modeled optimization problem, as shown in equation (15), is to minimize the number of physical machines needed to be used for deployment of virtual machines. The constraint shown in equation (16) indicates that the number of v-type virtual machines deployed in all physical machines must be greater than the required virtual machine pre-deployment number bv. In the problem modeling described above, the variables of the problem are related to the number of possible virtual machine deployment patterns, and further to the number of virtual machine types, rather than to the number of virtual machines. The size of the problem does not increase as the number of virtual machines increases.
The optimal solution of the optimization problem is an optimal virtual machine deployment scheme, and can be expressed as a vector x ═ x (x)1,…,xi…). Wherein each element x in the vectoriRepresenting the number of physical machines that need to be deployed in virtual machines according to mode i.
Step 403, solving the mathematical model of the optimization problem in step 402 according to the number of the virtual machines of each type to be pre-deployed, which is determined in step 401, to determine a final virtual machine deployment scheme:
the optimization problem mathematical model shown in equations (15) (16) is an integer programming problem in which each variable corresponds to a possible deployment pattern. In order to solve the problem, the invention firstly considers the linear relaxation problem of the problem, obtains an optimal solution of the relaxation problem, and finally obtains an approximate optimal solution of the original problem by rounding the optimal solution. Consider first the standard linear relaxation problem of the original problem, as follows:
min z＝cx (17)
where x is a column vector x1,x2,…,xn]TAnd c is a row vector [1,1, …,1 ]]B is a column vector [ b ]1,b2,…,bn]T. Each column vector a in matrix ai＝[ai1,…,aiv,…,aiV]TRepresenting a deployment scenario. The relaxed linear programming problem is very difficult to solve because even if the virtual machine type is limited, the number of possible deployment modes may be large, resulting in failure to form the coefficient matrix a in the solution process. The present invention then effectively solves this problem by using a simplex method based on a delay-column generation method.
Each step of iteration for solving the relaxed linear programming problem by using the simplex method mainly comprises the following four steps:
001) in one iteration, all variables will be divided into base variables xBNot less than 0 and the non-base variable x N0. Variable xBAnd xNThe values of which constitute the current feasible solution
The matrix a of the formula may then be divided into a ═ B | N]Wherein B is a base matrix containing the corresponding variable x in matrix ABAll columns of (a); n is a non-base matrix containing the corresponding variable x in matrix ABAll columns of (a). The cost row vector c in the final formula may be divided into c ═ cB,cN]In the form of (1), wherein cBComprises xBCoefficient corresponding to variable, cNComprises xNThe coefficients corresponding to the variables, and therefore the objective function (17) and the constraint (18), can be rewritten as:
002) next, reduced cost r is calculated for all non-base variablesN＝(cN-cBB-1N). If r isNAnd if the current feasible solution is more than or equal to 0, the current basic feasible solution is an optimal solution, and the current iteration is ended. Otherwise, the method can be implemented by satisfying a certain condition rNNon-base variables of < 0 are added and variables are used to optimize the value of the objective function. Therefore, the simplex method will select rNMiddle smallest element rjCorresponding non-base variable xjAs a new entry variable.
003) Calculating u-B-1Nj. If all the elements in u are negative, the current solved problem has no solution, and the current iteration is ended. If non-negative elements are present in u, then let
And assume that l satisfies
By dividing N in the original basis matrixjBy substitution with BlNew basis vectors may be formed and further based on u and θ*The base variables are updated.
004) And continuing the next iteration based on the generated new base vector and the base variable.
In the problem scenario considered by the present invention, there are a large number of feasible deployment modes of virtual machines, which results in the coefficient matrix a ═ B | N considered in the simplex method described above]Is very bulky. Therefore, in each iteration of the simplex method, r cannot be calculated by traversing all the columns in the matrix NN. To this end, this chapter calculates the required columns in N on demand by employing a delayed column generation technique in each iteration, rather than calculating all columns in N at once. Due to the vector cBIs 1. The invention therefore first calculates the vector p ═ cBB-1＝{p1,…,pvR, then solving each reduced costj＝(cj-pNj) The process of value can be minimized by minimizing (1-pN)j) Instead of this. This is equivalent to maximizing pNjIf the maximum value is less than or equal to 1, the reduced costs representing all the non-basic variables are non-negative, so that the chapter obtains an optimal solution. If the maximum element value is greater than 1, then this chapter finds the N corresponding to this maximum valuejAnd the corresponding non-base variable is taken as a new base variable.
Next, the problem to be solved in this chapter is to find the maximum pNjColumn j of (a). Since each column in the coefficient matrix a corresponds to a virtual machine deployment pattern, the value of each column must satisfy the constraint (14), and therefore the solution to maximize pNjThe problem for column j of (a) can be modeled as follows:
the above modeled problem can be viewed as a standard knapsack problem, where piCan be considered as the value of the article, riWhich can be considered the weight of the item, C can be considered the capacity of the backpack, and thus the above problem is equivalent to maximizing the sum of the values of the contained items without the weight of the contained items exceeding the backpack capacity C. Since the number V of virtual machine types in the scene considered in this chapter is limited, the above problem can be solved by a dynamic programming method.
The above-described embodiment is only one embodiment of the present invention, and it will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without limitation to the above-described embodiment of the present invention, based on the application and principle of the present invention, and therefore, the above-described embodiment is only preferred and not intended to be limiting.
Claims (8)
1. An online virtual resource allocation method based on virtual machine pre-deployment is characterized in that: the allocation method comprises the steps that firstly, an online virtual resource allocation framework based on virtual machine pre-deployment is constructed, and the framework realizes the organic combination of virtual machine deployment offline optimization and virtual machine deployment online decision through a virtual machine pre-deployment mode; secondly, designing an online virtual resource allocation flow based on the framework to realize rapid virtual resource allocation, and simultaneously, completing resource allocation by preferentially using a pre-deployed virtual machine to improve the utilization rate of physical resources of the data center after resource allocation; then, based on the framework and the process, a method for determining the optimal virtual machine pre-deployment scale under the resource allocation framework based on a queuing theory model is designed, and the offline resource allocation optimization effect is improved by reasonably determining the scale of offline optimization; finally, a mathematical model of the virtual machine pre-deployment scheme optimization problem in the offline optimization is constructed, the solving complexity of the virtual machine pre-deployment optimization problem in a large-scale environment is effectively reduced, the problem is solved, and finally the physical resource utilization rate is improved through reasonable deployment of heterogeneous virtual machines in the physical machine;
the method specifically comprises the following steps:
(step 1), constructing an online virtual resource allocation framework based on virtual machine pre-deployment;
step 2, designing an online virtual resource allocation flow based on the frame constructed in the step 1;
(step 3) determining the optimal virtual machine pre-deployment scale by using a queuing theory model based on the frames and the flows established in the steps (1) and (2), thereby improving the offline resource allocation optimization effect based on the virtual machine pre-deployment;
and (4) determining an optimal virtual machine deployment scheme according to the virtual machine pre-deployment scale determined in the step (3) so as to improve the physical resource utilization rate after the virtual machine deployment.
2. The method for allocating the online virtual resources based on the virtual machine pre-deployment according to claim 1, wherein: the online virtual resource allocation framework in the step (1) is divided into a static part and a dynamic part; in the static part, the deployment of the virtual machine in the physical machine is optimized in an off-line manner through the static pre-deployment of the virtual machine, so that the resource utilization rate of the physical machine is improved; in the dynamic part, the online dynamic deployment of the virtual machine is carried out according to the dynamic arrival of the resource allocation request, so that the quick response to the resource allocation request is realized; on one hand, the online virtual resource allocation framework realizes online resource allocation of dynamic arrival resource requests; and on the other hand, the offline optimization of resource allocation is realized through the optimization of the pre-deployment virtual machine.
3. The method for allocating the online virtual resources based on the virtual machine pre-deployment according to claim 1, wherein: the online virtual resource allocation process in step (2) includes the following steps:
(step 201), determining the type of virtual machine required for responding to the current arrival virtual resource allocation request;
(step 202), aiming at the type of the virtual machine corresponding to the resource allocation request, if the distributable deployed type virtual machine exists in the system, distributing the corresponding virtual machine for the resource allocation request, thereby completing the virtual resource allocation;
(step 203), if the system does not have the distributable deployed virtual machine of the type currently, dynamically deploying one virtual machine of the type, and distributing the virtual machine for the current resource distribution request, thereby completing the virtual resource distribution.
4. The method for allocating the online virtual resources based on the virtual machine pre-deployment according to claim 1, wherein: the method for determining the optimal virtual machine pre-deployment scale in the step (3) comprises the following steps:
(step 301), carrying out statistical analysis on the dynamic arrival condition of the resource allocation request, and modeling the deployment demand of each type of virtual machine on the basis of the statistical analysis;
(step 302), constructing a pre-deployment scale optimization model based on a queuing theory, and determining the scale of the pre-deployment virtual machine and the functional relationship between the utilization rates of the pre-deployment virtual machine;
(step 303) determining the optimal virtual machine pre-deployment scale by using a binary search method according to the determined functional relation in the step 302.
5. The method for allocating the online virtual resources based on the virtual machine pre-deployment according to claim 1, wherein: the step (step 4) comprises the following steps:
(step 401), determining the number of each type of virtual machine required to be pre-deployed according to the optimal virtual machine pre-deployment scale determined in the step (303) of claim 4;
(step 402), constructing a mathematical model of a virtual machine pre-deployment scheme optimization problem;
(step 403), solving the mathematical model of the optimization problem (step 402) according to the number of the virtual machines of each type to be pre-deployed (step 401), and determining a final virtual machine deployment scheme.
6. The method for optimizing the virtual machine pre-deployment scheme according to claim 5, wherein: in the step 501, when a mathematical model of the virtual machine pre-deployment scheme optimization problem is constructed, the limited virtual machine types of the data center are considered, so that a "virtual machine deployment mode" is defined based on the virtual machine types, and the mathematical modeling is performed on the virtual machine pre-deployment scheme optimization problem based on the virtual machine type; the "virtual machine deployment mode" is defined as follows:
definition 1: the virtual machine deployment pattern represents a configuration pattern of a physical machine that indicates which virtual machine types and how many of these types of virtual machines are deployed in the physical machine. The invention uses a vector A containing V dimensionsi＝[ai1,…,aiv,…,aiV]To represent a deployment pattern i, where aivRepresenting the number of v-type virtual machines that can be deployed in a physical machine configured according to mode i.
7. The method for optimizing the virtual machine pre-deployment scheme according to claim 5, wherein: in the step 403, the mathematical model of the optimization problem in the step 402 is solved by a simplex method based on a delay column generation method.
8. The method for allocating the online virtual resources based on the virtual machine pre-deployment according to claim 1, wherein: the principle of the distribution method is that an online calculation method and an offline method are organically combined, so that the online method is used for realizing the rapid deployment of the virtual machine, and the offline method is used for performing the offline optimization of the deployment of the virtual machine.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910625142.XA CN111309472A (en) | 2019-07-11 | 2019-07-11 | Online virtual resource allocation method based on virtual machine pre-deployment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910625142.XA CN111309472A (en) | 2019-07-11 | 2019-07-11 | Online virtual resource allocation method based on virtual machine pre-deployment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111309472A true CN111309472A (en) | 2020-06-19 |
Family
ID=71156378
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910625142.XA Pending CN111309472A (en) | 2019-07-11 | 2019-07-11 | Online virtual resource allocation method based on virtual machine pre-deployment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111309472A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112363808A (en) * | 2020-11-27 | 2021-02-12 | 湖南女子学院 | Virtual machine dynamic deployment method based on double-vernier control mechanism |
| CN113722660A (en) * | 2021-08-31 | 2021-11-30 | 西南科技大学 | Optimized configuration method for ordinary-speed single-track railway tunnel slag-tapping transport machinery |
| CN113965574A (en) * | 2021-09-27 | 2022-01-21 | 西安交通大学 | Cloud service data center backup virtual resource scheduling method and device |
| WO2022262511A1 (en) * | 2021-06-18 | 2022-12-22 | 华为技术有限公司 | Service data processing method and device |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8914795B2 (en) * | 2011-01-31 | 2014-12-16 | International Business Machines Corporation | Determining an allocation configuration for allocating virtual machines to physical machines |
| CN108089914A (en) * | 2018-01-18 | 2018-05-29 | 电子科技大学 | A kind of cloud computing deploying virtual machine algorithm based on energy consumption |
-
2019
- 2019-07-11 CN CN201910625142.XA patent/CN111309472A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8914795B2 (en) * | 2011-01-31 | 2014-12-16 | International Business Machines Corporation | Determining an allocation configuration for allocating virtual machines to physical machines |
| CN108089914A (en) * | 2018-01-18 | 2018-05-29 | 电子科技大学 | A kind of cloud computing deploying virtual machine algorithm based on energy consumption |
Non-Patent Citations (1)
| Title |
|---|
| 史济源: "面向复杂科学计算应用的虚拟资源柔性管理机制研究", 《中国优秀博士学位论文全文数据库信息科技辑》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112363808A (en) * | 2020-11-27 | 2021-02-12 | 湖南女子学院 | Virtual machine dynamic deployment method based on double-vernier control mechanism |
| CN112363808B (en) * | 2020-11-27 | 2024-09-13 | 湖南女子学院 | Virtual machine dynamic deployment method based on double-cursor control mechanism |
| WO2022262511A1 (en) * | 2021-06-18 | 2022-12-22 | 华为技术有限公司 | Service data processing method and device |
| CN113722660A (en) * | 2021-08-31 | 2021-11-30 | 西南科技大学 | Optimized configuration method for ordinary-speed single-track railway tunnel slag-tapping transport machinery |
| CN113965574A (en) * | 2021-09-27 | 2022-01-21 | 西安交通大学 | Cloud service data center backup virtual resource scheduling method and device |
| CN113965574B (en) * | 2021-09-27 | 2022-07-12 | 西安交通大学 | Cloud service data center backup virtual resource scheduling method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Kaur et al. | Container-as-a-service at the edge: Trade-off between energy efficiency and service availability at fog nano data centers | |
| Alkayal et al. | Efficient task scheduling multi-objective particle swarm optimization in cloud computing | |
| Rekha et al. | Efficient task allocation approach using genetic algorithm for cloud environment | |
| CN111309472A (en) | Online virtual resource allocation method based on virtual machine pre-deployment | |
| Nabi et al. | DRALBA: Dynamic and resource aware load balanced scheduling approach for cloud computing | |
| Barbosa et al. | Dynamic scheduling of a batch of parallel task jobs on heterogeneous clusters | |
| CN104239141A (en) | Task optimized-scheduling method in data center on basis of critical paths of workflow | |
| CN108572873A (en) | A kind of load-balancing method and device solving the problems, such as Spark data skews | |
| Tantalaki et al. | Pipeline-based linear scheduling of big data streams in the cloud | |
| Thaman et al. | Green cloud environment by using robust planning algorithm | |
| CN114610474B (en) | Multi-strategy job scheduling method and system under heterogeneous supercomputing environment | |
| CN115134371A (en) | Scheduling method, system, equipment and medium containing edge network computing resources | |
| CN111159859B (en) | Cloud container cluster deployment method and system | |
| CN109815009A (en) | Scheduling of resource and optimization method under a kind of CSP | |
| CN106407007B (en) | Cloud resource configuration optimization method for elastic analysis process | |
| Filelis-Papadopoulos et al. | Towards simulation and optimization of cache placement on large virtual content distribution networks | |
| CN106802822A (en) | A kind of cloud data center cognitive resources dispatching method based on moth algorithm | |
| CN108958919A (en) | More DAG task schedule expense fairness assessment models of limited constraint in a kind of cloud computing | |
| CN114860417B (en) | Multi-core neural network processor and multi-task allocation scheduling method for same | |
| CN111522637A (en) | Storm task scheduling method based on cost benefit | |
| Li et al. | On scheduling of high-throughput scientific workflows under budget constraints in multi-cloud environments | |
| CN115098240B (en) | Multiprocessor application scheduling method and system and storage medium | |
| CN110990130A (en) | Reproducible self-adaptive computation unloading layered service quality optimization method | |
| Cao et al. | Online cost-rejection rate scheduling for resource requests in hybrid clouds | |
| CN115543582A (en) | Method, system and equipment for unified scheduling of super computing power network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20200619 |