CN112269632B - Scheduling method and system for optimizing cloud data center - Google Patents
Scheduling method and system for optimizing cloud data center Download PDFInfo
- Publication number
- CN112269632B CN112269632B CN202011023815.3A CN202011023815A CN112269632B CN 112269632 B CN112269632 B CN 112269632B CN 202011023815 A CN202011023815 A CN 202011023815A CN 112269632 B CN112269632 B CN 112269632B
- Authority
- CN
- China
- Prior art keywords
- virtual machine
- scheduling
- machine
- task
- migration
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 35
- 230000005012 migration Effects 0.000 claims abstract description 81
- 238000013508 migration Methods 0.000 claims abstract description 81
- 230000000737 periodic effect Effects 0.000 claims abstract description 14
- 238000004364 calculation method Methods 0.000 claims description 14
- 238000012544 monitoring process Methods 0.000 claims description 13
- 238000011156 evaluation Methods 0.000 claims description 9
- 230000006870 function Effects 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 5
- 230000005059 dormancy Effects 0.000 claims description 3
- 238000012163 sequencing technique Methods 0.000 claims description 2
- 238000001514 detection method Methods 0.000 claims 1
- 238000005265 energy consumption Methods 0.000 description 22
- 230000007958 sleep Effects 0.000 description 10
- 238000007726 management method Methods 0.000 description 4
- 230000003068 static effect Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000005057 refrigeration Methods 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000001816 cooling Methods 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 230000002068 genetic effect Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000002922 simulated annealing Methods 0.000 description 1
- 239000000725 suspension Substances 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
- G06F1/26—Power supply means, e.g. regulation thereof
- G06F1/32—Means for saving power
- G06F1/3203—Power management, i.e. event-based initiation of a power-saving mode
- G06F1/3234—Power saving characterised by the action undertaken
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5072—Grid computing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5077—Logical partitioning of resources; Management or configuration of virtualized resources
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/104—Peer-to-peer [P2P] networks
- H04L67/1044—Group management mechanisms
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45595—Network integration; Enabling network access in virtual machine instances
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/5011—Pool
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention discloses a scheduling method and a scheduling system for optimizing a cloud data center. Aiming at the problem of low efficiency of a resource scheduling mode of the existing cloud computing data center, the scheduling method and system comprise a streaming virtual machine deployment scheme and a periodic dynamic virtual machine scheduling scheme. The method and the system can monitor the resource utilization rate of the server and the virtual machine, and deploy the virtual machine by classifying the priorities of different tasks. Under the condition of guaranteeing QoS of users, the migration or inert non-migration of small part of virtual machines is performed periodically, the utilization rate of various resources of the server cluster can be effectively improved, and unnecessary on-off operation and corresponding energy loss of the server are avoided.
Description
Technical Field
The invention belongs to the field of resource and energy consumption management of cloud data centers, and particularly relates to a method and a system for deploying virtual machines and migrating the virtual machines.
Background
In the last decade, the energy consumption of data centers has attracted considerable attention in the field of information technology. America Amazon corporation in the recent report stated that its company's data center energy consumption was over 40% of the total economic overhead in a period of three years. From an economic point of view, if the power consumption of the data center continues to increase at the current stage, the future electricity fee can easily exceed the hardware investment of the data center itself. At the same time, excessive energy consumption can increase the load on the power plant and power plants that are not clean energy can have a further impact on the environment.
The energy consumption of the data center is mainly distributed as follows: computing clusters, refrigeration systems, and others. With the other total 5% being comprised primarily of light illumination and UPS. The energy consumption of the refrigeration system accounts for 40 percent, and the energy consumption of the computing cluster accounts for 55 percent. At present, a part of novel small and medium-sized data centers greatly reduce the total power consumption of a refrigerating system by adopting a free cooling system, and at most can be reduced to half of the total power consumption of the whole year. Therefore, the maximum power consumption of the computing cluster can reach 75% of the overall power consumption of the data center. In order to reduce the power consumption of the data center, the technology for optimizing the energy consumption of the computer group is very critical. The energy consumption of the computing cluster is mainly divided into static energy consumption and dynamic energy consumption. The static energy consumption refers to that under the state that a physical machine does not execute a specific task and is in standby, the basic power consumption of the single machine of the physical machine accounts for 40% of the total power consumption of the physical machine; the dynamic energy consumption is the power consumption increased on the basis of the static energy consumption when the physical machine executes the task, and the dynamic energy consumption mainly comprises a CPU and a RAM, and the power consumption of the network card and the hard disk is relatively less.
In cloud computing, virtualization is a common resource management technique. Virtualization abstracts the various entities of a computer, such as a CPU, RAM, and hard disk. In the operation task, different tasks are operated on different virtual machines, and different virtual machines can be selectively operated on the same or different physical machines.
The cloud computing data center cluster is virtualized, when a task arrives, a system can create a virtual machine with a corresponding size, and the task can finish running in the virtual machine.
Regarding resource management, the virtual machine deployment mostly adopts offline methods such as genetic algorithm, simulated annealing and the like, and the optimization target is that the load between physical machines is balanced, and the single machine has higher load rate as much as possible.
Regarding power consumption, a mainstream solution is to migrate and merge virtual machines, sleep or shut down part of physical machines to reduce power consumption.
The main problems of the prior art solutions are the following,
1. the control module often migrates and merges the virtual machines according to the current resource utilization rate. After the migration merge action is completed, the current server may sleep or be turned off. However, often just after the server is shut down, a new task arrives soon and the server needs to be re-activated. In the whole process of closing and opening, the migration, system suspension and starting of the virtual machine occupy CPU resources and time, and cause more energy consumption. In other words, the timing of the decision to merge is also of greater impact on energy consumption.
2. Most of the existing task scheduling algorithms are offline in order to guarantee higher cluster resource utilization. And at the beginning of a scheduling algorithm, submitting all tasks needing to be run to a scheduling module. The scheduling module creates a virtual machine according to the size of the resources required by the task and distributes the virtual machine to the physical machines with enough resources in the cluster for operation. In order to achieve better utilization rate of cluster resources after a new task arrives, all running virtual machines need to be redeployed, and the redeployment of all virtual machines has high energy consumption cost and can possibly affect the user service quality of part of the tasks. For example, existing hot migration techniques for virtual machines may cause short-time interruption of tasks in the virtual machine when migrating the virtual machine, and may particularly have a great impact on some tasks that provide web-services.
3. The current mainstream online algorithm mainly uses the utilization rate of cpu as a scheduling reference unit, because cpu is the component with the largest energy consumption in a physical single machine. If only cpu is used as the basic scheduling unit, the cpu occupancy rate of part of the physical machines is higher, the memory or network utilization rate is lower, and the internal resource utilization is very unbalanced, so that the resource waste is caused. There is still room for, for example, machine cpu resources, but the memory and network are full. The server has three physical resources of CPU, ram and net, and a reasonable deployment mode should be placed by comprehensive consideration of multiple variables of CPU, memory and network resources.
4. The offline virtual machine deployment algorithm requires longer computation time than the online algorithm, and requires recalculation of the deployment scheme for all virtual machines each time a new task arrives.
Disclosure of Invention
Aiming at the problem of low efficiency of a resource scheduling mode of the existing cloud computing data center, the invention provides an online virtual machine deployment scheme for optimizing the resources and energy consumption of the cloud computing data center and a periodic dynamic virtual machine scheduling and converging method. The method and the system can monitor the resource utilization rate of the server and the virtual machine, and can periodically and rapidly adjust the deployment scheme of the prior virtual machine while guaranteeing the service quality of the user by classifying the priorities of different tasks, thereby improving the resource utilization rate of the local server and reducing the total power consumption.
The invention discloses a scheduling method for optimizing cloud data center resources, which comprises a streaming virtual machine deployment step and a periodic dynamic virtual machine scheduling step, wherein an arrival task is deployed according to the streaming virtual machine deployment step; judging whether virtual machine scheduling is carried out or not according to the periodic dynamic virtual machine scheduling step in each set scheduling period, and executing according to a judging result;
the stream virtual machine deployment step comprises the following steps:
1) When the task arrives, classifying the task according to the task type, and putting the task into a corresponding task pool to wait for scheduling; the task types include a time delay sensitive type and a non-time delay sensitive type;
2) Acquiring the current CPU, ram and net resource conditions of each physical machine, and summarizing the ratio of the resource occupation of each physical machine in on state; performing simulated placement, namely, after the current virtual machine is placed into each physical machine with on state, summarizing the occupation ratio of each left resource of each physical machine; selecting a first candidate machine with the optimal resource occupation ratio to carry out new task placement; if no candidate machine exists, selecting any off physical machine to place a new task or returning the task to a task pool for waiting continuously according to the task type;
the periodic dynamic virtual machine scheduling step comprises the following steps:
a) Obtaining the load condition of each physical machine, obtaining the sum of all resource demands which will reach the virtual machine in the next period, generating a virtual machine migration plan,
b) Judging whether the virtual machine migration plan is executed in the scheduling period; if the virtual machine is executed, the virtual machine is migrated according to a virtual machine migration plan; otherwise, the scheduling of this scheduling period ends.
The invention also discloses a dispatching system for optimizing the cloud data center resources, which comprises a monitoring module, a prediction module, a dispatching module, an evaluation module and a task classification module;
the task classification module puts the new task into a corresponding task pool according to the classification;
the monitoring module is used for monitoring the working condition of the physical machines and transmitting the cpu, ram and net resource condition of each physical machine to the scheduling module;
the prediction module is used for predicting the load condition of the virtual machine in the next period, namely, each resource requirement of the virtual machine will be reached in the next period, and transmitting the prediction information to the scheduling module;
the scheduling module acquires current resource duty ratio information of the physical machine from the monitoring module, and performs tasks in the task pool on the physical machine;
the scheduling module also obtains prediction information of future load conditions of the virtual machine from the prediction module; generating a virtual machine scheduling plan according to the current resource duty ratio information and the prediction information of the physical machine; the scheduling module judges whether scheduling is carried out according to the virtual machine scheduling plan power consumption calculation result fed back by the evaluation module, and performs scheduling on the condition needing scheduling;
the evaluation module acquires a virtual machine scheduling plan generated by the scheduling module, and calculates power consumption generated by the virtual machine migration plan; and feeding back the calculation result to the scheduling module.
The invention can provide a dynamic resource management method for a cloud data center. The method has the advantages that the migration or inert non-migration of a small part of virtual machines is performed periodically, the utilization rate of various resources of the server cluster can be effectively improved, and unnecessary on-off operation and corresponding energy loss of the server are avoided.
Drawings
FIG. 1 is a diagram of a physical machine (including the remaining resource ratios);
FIG. 2 is a schematic diagram of an optimized cloud data center scheduling system of the present invention.
Detailed Description
The invention is further illustrated and described below in connection with specific embodiments. The technical features of the embodiments of the invention can be combined correspondingly on the premise of no mutual conflict. In the present invention, the physical machine and server concepts and attributes are equivalent unless otherwise specified.
The method comprises a streaming virtual machine deployment step and a periodic dynamic virtual machine scheduling step, wherein an arrival task is deployed according to the streaming virtual machine deployment step; and judging whether to schedule the virtual machine or not according to the periodic dynamic virtual machine scheduling step in each set scheduling period, and executing according to a judging result.
Task vm i Expressed as: vm (vm) i ＝(s i ，e i ，d i ，type，<cpu，ram，net>，m id )
Wherein s is i Refer to the task start time; e, e i Means the task end time; d, d i The estimated required time of the task; type refers to a task type, type is b (batch-service, i.e., non-latency sensitive task) or w (web-service, i.e., latency sensitive task);<cpu，ram，net>the resources required by the virtual machine where the task is located are respectively corresponding to: cpu refers to the number of cpu cores required by the virtual machine, ram refers to the memory capacity required by the virtual machine, and net refers to the network bandwidth required by the virtual machine; m is m id Representing the physical machine number where the virtual machine is located.
The time delay sensitive task refers to a task of which the type is not interrupted as much as possible in the running process;
non-latency sensitive tasks refer to tasks that can be interrupted during operation, then stored as snapshots (snapshot), and later can wake up to resume operation.
As shown in FIG. 1, the resources of each physical machine are divided into three<cpu，ram，net>Cpu is the number of cpus of the physical machine, ram is the memory capacity, and net is the network bandwidth of the physical machine. The physical machine has two states: on and off. on means that the current physical machine is in an operating state (highPower consumption), off means that the current physical machine is in a sleep or off state (low or no power consumption). The two states can be switched with each other, and the energy loss in the switching can be respectively described as switching from operation (on) to dormancy (off) off ＝t off *e off ，t off Refers to the time required to switch from running to sleep; e, e off The power consumption of the physical machine in unit time from operation to sleep state is referred; switching from sleep (off) to sleep (on) Switch on ＝t on *e on ，t on Refers to the time required to switch from sleep to run, e on Refers to the power consumption per unit time of the physical machine switching from sleep to run state.
1. Online virtual machine deployment scheme for cloud computing data center
The task is submitted to the cloud by the user, and the corresponding virtual machine resource requirement is submitted at the same time. The scheduling module acquires resource indexes required by the virtual machine, wherein the resource indexes comprise cpu, ram and net. Cpu refers to the number of cores of CPU required by the virtual machine; ram refers to the memory capacity required by the virtual machine; net refers to the network bandwidth required by a virtual machine. The tasks are classified according to the task types submitted by the user, and are mainly classified into time delay sensitive types and non-time delay sensitive types. Latency sensitive type is typically referred to as web-service type tasks such as shopping websites, clothing websites, personal blogs, and the like. Such tasks are generally long and very ductile, and migration of such virtual machines results in a brief service pause, so such tasks are unsuitable for migration. In contrast, non-latency sensitive multi-fingered computing-like tasks (batch-service), which are relatively latency insensitive, are mostly read-write intensive (I/O-intensive) or computation intensive (cpu-intensive). Such tasks typically have a specific calculated deadline (readline), i.e., they are completed within a specified time.
The scheduling module gathers the ratio of the left resources of each physical machine after obtaining the current CPU, ram and net resource condition of each physical machine from the monitoring module.
And then, the scheduling module performs simulated placement, namely, after the current virtual machine is placed into each physical machine, the occupation ratio of the left resources of each physical machine is summarized. And selecting the machine with the optimal resource duty ratio as the first candidate machine. After the task is placed, the same calculation is performed for the next arriving task.
2. Periodic dynamic virtual machine dispatch join
The control module obtains the load condition of each physical machine and the load information of the virtual machine from the monitoring module at the beginning of each time period (e.g., one hour). And meanwhile, the scheduling module acquires prediction information of future load conditions of the virtual machine from the prediction module, and decides whether to schedule the virtual machine in the time period according to the information.
If scheduled improperly or too frequently, unnecessary virtual machine migration overhead may cause power consumption of the entire cluster to rise. The scheduling module should be as inert as possible to determine whether to schedule in the current period. If scheduling is needed, the scheduling module generates a scheduling plan, and then migrates the designated virtual machine to the target physical machine, and then closes or sleeps part of the physical machines, thereby improving the resource utilization rate of the cluster and reducing the overall power consumption of the cluster.
The scheduling module traverses all non-delay sensitive type tasks, selects a part of virtual machines meeting the conditions and adds the virtual machines to the migration plan. For the tasks of the time delay sensitive type, the virtual machine migration can cause service interruption of the tasks because the tasks are very sensitive to the time delay, so that the scheduling module does not preferentially perform migration operation on the virtual machines of the tasks.
And after the scheduling module generates a partial virtual machine migration plan of the time interval, calculating the power consumption generated by the scheduling. The virtual machine migration cost function includes: the power consumption of CPU and ram consumed by virtual machine migration, and the power consumption of a network in the virtual machine migration process. After the migration of the virtual machine group is finished, the server enters dormant power consumption and awakening power consumption. The virtual machine migrates the reward function, i.e. the power consumption that can be reduced after migration. Migration is too frequent, which causes great network and energy consumption expense, so that scheduling should be performed according to the principle of lazy scheduling, that is, only virtual machine migration plans which can bring gain are executed.
If the reward obtained after migration is less than the cost of migration, the virtual machine migration plan with excessive power consumption will not be executed. Instead, virtual machine migration plans through evaluation will be executed immediately. And after the virtual machine is migrated, sleeping or closing the appointed server. This periodic schedule ends.
The most main optimization objective of the scheduling algorithm is to increase the comprehensive resource utilization rate of the physical machine by dormancy or shutting down the unnecessary physical machine. .
As shown in fig. 2, a detailed procedure of the dynamic resource scheduling method of the present invention is described next based on a legend.
When the task arrives, the tasks are respectively put into the corresponding task pools to wait for scheduling according to the types of the tasks. At this time, the scheduling module obtains the ratio of the left resources of each physical machine from the physical machine monitoring module, u cpu ，u ram ，u net 。
Wherein the method comprises the steps ofRepresenting the resources left by the current cpu; />Representing the memory resources left by the current physical machine; />Representing the network resources left by the current physical machine;
the calculation is performed for each physical machine i with all states on: alpha, beta, gamma are weight coefficients, v cpu CPU resources (e.g., CPU cores) occupied by virtual machines, v ram Memory resources (e.g., memory size), v occupied by virtual machines net Is a network resource of the virtual machine. After the calculation is completed, the pm_score is used for all physical machines i The value of (pm_score) i > 0) ranking from large to small to generate a candidate server list. If the task is a web-service class task, the virtual machine is directly placed into the physical machine of candidate server list rank 1. If the candidate list is empty, selecting any physical machine with the off state and placing the current virtual machine into the physical machine.
If the task is a non-delay sensitive task, in all physical machines in the current state of on, no one of the physical machines has enough resources to deploy the virtual machine, and then a slot is calculated for the task in the virtual machine, wherein slot=e i -s i -d i . If the slot is more than or equal to 0, the task is put into a non-delay sensitive task pool to wait continuously; if the slot is less than 0, the task is prioritized and treated as a latency sensitive task.
Each time a new batch-service class task is returned to the batch pool, the scheduling module performs the following calculationWhen->When the total demand of the tasks cpu in the batch pool exceeds 50% of the cpu resources of a single physical machine, any physical machine with off state is selected to be started, and the virtual machines corresponding to the tasks in all the batch pool are placed on the physical machine.
When the scheduling module detects that the slot of a certain task in the batch pool is less than 1, any physical machine with off state is selected to start, and virtual machines corresponding to the tasks in all the batch pools are placed on the physical machine.
The periodic virtual machine migration plan is as follows:
a time-delay sensitive task (web-service) may be described as < cpu_use, ram_use, network_use, c_mig, duration >. Three parameters of cpu_usage, ram_usage and network_usage in the time delay sensitive task refer to the usage rate of cpu, the usage rate of memory and the usage rate of network bandwidth in the current time period, and cpu_usage, ram_usage and net_usage are variables; migration cost
τ is typically a constant greater than 1, and should not migrate web-service type tasks as much as possible so as not to affect user quality of service; e, e m The power consumption of the physical machine and the network in unit time during virtual machine migration is realized;the time required to migrate the virtual machine; c mig At the cost of migration; the duration (duration) is typically a constant.
Non-latency sensitive tasks (batch-services) can be described as<cpu_usage，ram_usage，network_usage，c mig ，duration，deadline>. The cpu_use and ram_use and network in the Batch-service task are typically very small amounts of change and can be considered invariant. Migration costC compared to web-services mig Without coefficient tau, a smaller migration cost can be obtained under the same condition, and the priority of migration non-delay sensitive type is higher than that of delay sensitive type; the duration of the non-delay sensitive type is typically a finite time.
At the beginning of each time period, the scheduling module obtains the current resource duty ratio information of the physical machine from the monitoring module. Counting the current cpu, memory and the sum of all resources of the network of all physical machines in on state:
the scheduling module obtains the CPU, memory and network resource demands of the virtual machine to be reached in the next period from the prediction module, and obtains the resources of the virtual machine to be reached in the next period from the prediction moduleSum of demandRespectively calculate r 1 -r′ 1 ，r 2 -r′ 2 ，r 3 -r′ 3 If one of the entries is less than 0, then this cycle does not perform virtual machine migration. If r 1 -r′ 1 ，r 2 -r′ 2 ，r 3 -r′ 3 All are greater than or equal to 0, and the scheduling module starts scheduling plan generation:
the calculation is performed for each physical machine i with all states on: three corresponding lists list_machine_CPU, list_machine_RAM, list_machine_NET are generated according to the obtained scores from large to small. Finally, a score is calculated for each physical machine:
the resulting pm_score was then used to determine i The physical machine list list_machine is generated in order from big to small.
Physical machine pm 1 Rank first in list_machine, therefore, move it into migration manifest, pm 1 Based on virtual machine scores:sequencing and corresponding to the pm_CPU_score according to the highest CPU/ram/net item in the current virtual machine i ，pm_RAM_score i ，pm_NET_score i Matching the physical machines before migration. For example vm score i The highest score is +.>Then at pm_CPU_score i And selecting physical machines meeting the resource requirements of the virtual machines according to the list sequence as migration objects. And so on until the physical machine pm 1 All virtual machines on the target migration server are matched. The scheduling module adds the migration scheme of the virtual machine in the physical machine to the total scheduling plan of the current time period. Then re-updating all the physical machines pm_score in the list_machine i . According to the method, the physical machines ranked 1 in the current list-machine attempt to generate a virtual machine migration scheme until all virtual machines in a certain physical machine cannot be migrated to all physical machines in the current state on, and the scheduling module calculates and terminates.
At this time, the scheduling module calculates the power consumption generated by the scheduling. The virtual machine migration cost function includes: the power consumption of CPU and ram consumed by virtual machine migration, and the power consumption of a network in the virtual machine migration process. After the migration of the virtual machine group is finished, the server enters dormant power consumption and awakening power consumption. The virtual machine migrates the reward function, i.e. the power consumption that can be reduced after migration. Migration is too frequent, which causes great network and energy consumption expense, so that scheduling should be performed according to the principle of lazy scheduling, that is, only virtual machine migration plans which can bring gain are executed.
If the reward obtained after migration is less than the cost of migration, the virtual machine migration plan with excessive power consumption will not be executed. Instead, virtual machine migration plans through evaluation will be executed immediately.
The virtual machine migration rewards function is defined as:and i corresponds to all virtual machines needing to be migrated in the migration plan, and i corresponds to a physical machine needing to be closed.
The foregoing examples illustrate only a few embodiments of the invention and are described in detail herein without thereby limiting the scope of the invention. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the invention, which are all within the scope of the invention. Accordingly, the scope of protection of the present invention is to be determined by the appended claims.
Claims (8)
1. A scheduling method for optimizing a cloud data center is characterized by comprising the following steps of:
the method comprises a streaming virtual machine deployment step and a periodic dynamic virtual machine scheduling step, wherein an arrival task is deployed according to the streaming virtual machine deployment step; judging whether virtual machine scheduling is carried out or not according to the periodic dynamic virtual machine scheduling step in each set scheduling period, and executing according to a judging result;
the stream virtual machine deployment step comprises the following steps:
1) When the task arrives, classifying the task according to the task type, and putting the task into a corresponding task pool to wait for scheduling; the task types include a time delay sensitive type and a non-time delay sensitive type;
2) Acquiring the current CPU, ram and net resource conditions of each physical machine, and summarizing the ratio of the resource occupation of each physical machine in on state; performing simulated placement, namely, after the current virtual machine is placed into each physical machine with on state, summarizing the occupation ratio of each left resource of each physical machine; selecting a physical machine with the optimal resource ratio as a first candidate machine for new task placement; if no candidate machine exists, selecting any off physical machine to place a new task or returning the task to a task pool for waiting continuously according to the task type;
the periodic dynamic virtual machine scheduling step comprises the following steps:
a) The method comprises the steps of obtaining the load condition of each physical machine, obtaining the sum of all resource requirements of the virtual machine to be reached in the next period, and judging whether to generate a virtual machine migration plan, wherein the virtual machine migration plan is generated only when the sum of all resources of the current cpu, the memory and the network of the physical machine with on state is not smaller than the sum of the corresponding resource requirements of the virtual machine to be reached in the next period;
b) Judging whether the virtual machine migration plan is executed in the scheduling period; if the virtual machine is executed, the virtual machine is migrated according to a virtual machine migration plan; otherwise, the scheduling of this scheduling period ends.
2. The method for optimizing cloud data center as claimed in claim 1, wherein the task in said step 1) is denoted as vm i It is expressed as: vm (vm) i ＝(s i ，e i ，d i ，type，<cpu，ram，net>，m id )
Wherein s is i Refer to the task start time; e, e i Means the task end time; d, d i The estimated required time of the task; type refers to task type;<cpu，ram，net>the resources required by the virtual machine where the task is located are respectively corresponding to: cpu refers to the number of cpu cores required by the virtual machine, ram refers to the memory capacity required by the virtual machine, and net refers to the network bandwidth required by the virtual machine; m is m id Representing the physical machine number where the virtual machine is located.
3. The method for optimizing cloud data center scheduling according to claim 1 or 2, wherein the step 2) specifically comprises:
acquiring the ratio of the remaining resources of each physical machine with on state: u (u) cpu ，u ram ，u net ，
Wherein the method comprises the steps ofRepresenting the resources left by the current cpu; />Representing the memory resources left by the current physical machine; />Representing the network resources left by the current physical machine;
the calculation is performed for each physical machine i with all states on: alpha, beta, gamma are weight coefficients, v cpu CPU resource, v occupied by virtual machine ram Memory resources, v, occupied by virtual machines net For network resource of virtual machine, after calculation is completed, according to pm_score i To rank all physical machines from big to small, wherein pm_score generates a candidate server list i ＞0；
For the time delay sensitive task, directly placing the virtual machine into the physical machine of the candidate server list rank 1; if the candidate list is empty, selecting any physical machine with off state and placing the current virtual machine into the physical machine;
for a non-latency sensitive task, a time tolerance slot is calculated for this task, which is specifically defined as: slack=e i -s i -d i The method comprises the steps of carrying out a first treatment on the surface of the If the slot is more than or equal to 0, the task is put into a non-delay sensitive task pool to wait continuously; if the slot is less than 0, the task is prioritized and treated as a latency sensitive task.
4. A method of scheduling an optimized cloud data center as claimed in claim 3, wherein each time a new non-latency sensitive task is returned to the non-latency sensitive task pool, the following calculation is performedWhen->When the virtual machine is started, any physical machine with off state is selected, and virtual machines corresponding to tasks in all non-time delay sensitive task pools are placed on the physical machine; wherein m is cpu Is a CPU resource of a single physical machine;
when the detection that the slot of a certain task in the non-delay sensitive task pool is less than 1, any physical machine with off state is selected to start, and virtual machines corresponding to the tasks in all batch pools are placed on the physical machine.
5. The method for optimizing cloud data center scheduling according to claim 1, wherein the step a) specifically includes:
at the beginning of each time period, the scheduling module acquires the current resource duty ratio information of the physical machine from the monitoring module, and counts the total r of all resources of the current cpu, the memory and the network of the physical machine with on state 1 ，r 2 ，r 3 The scheduling module obtains the resource requirements r 'of CPU, memory and network of the virtual machine, which are reached in the next period, from the prediction module' 1 ，r′ 2 ，r′ 3 Respectively calculate r 1 -r′ 1 ，r 2 -r′ 2 ，r 3 -r′ 3 If one of the items is smaller than 0, the period does not generate a scheduling plan, namely virtual machine migration is not carried out; if r 1 -r′ 1 ，r 2 -r′ 2 ，r 3 -r′ 3 And the two are all greater than or equal to 0, and the scheduling module starts to generate a scheduling plan.
6. The method for optimizing cloud data center scheduling according to claim 1 or 5, wherein the generating a scheduling plan specifically includes:
for each physical machine i with all states on, performing calculation: generating three corresponding lists of list_machine_CPU, list_machine_RAM and list_machine_NET from large to small according to the obtained scores;
finally, calculate the score for each physical machine： The resulting pm_score was then used to determine i Generating a physical machine list_machine according to the order from big to small;
physical machine pm 1 Ranking first in list_machine, move it into migration manifest, pm 1 Based on virtual machine scores:sequencing and corresponding to the pm_CPU_score according to the highest CPU/ram/net item in the current virtual machine i ，pm_RAM_score i ，pm_NET_score i Matching the physical machine before migration; until the physical machine pm 1 All virtual machines on the target migration server are matched with the target migration server;
the scheduling module adds the migration scheme of the virtual machine in the physical machine to the total scheduling plan of the current time period, and then updates all the physical machines pm_score in the list_machine again i The method comprises the steps of carrying out a first treatment on the surface of the And then generating a virtual machine migration scheme for the physical machine ranked 1 in the current list-machine until all virtual machines in a certain physical machine cannot be migrated to all physical machines in the current state of on, and stopping calculation by a scheduling module to obtain a final scheduling plan.
7. The method for optimizing cloud data center scheduling according to claim 1, wherein the step B) specifically includes:
calculating the power consumption generated by the virtual machine migration plan, namely calculating a virtual machine migration cost function, wherein the virtual machine migration cost function comprises the power consumption of cpu and ram consumed by virtual machine migration, the power consumption of a network in the virtual machine migration process, and the power consumption of a physical machine entering dormancy and the power consumption of awakening after the virtual machine group migration is finished; calculating a migration rewarding function of the virtual machine, namely the power consumption which can be reduced after migration;
if the reward obtained after migration is less than the cost of migration, the virtual machine migration plan will not be executed; in contrast, the virtual machine migration plan through evaluation will be executed immediately, and after the virtual machine migration is completed, the specified physical machine is asleep or turned off, and the periodic scheduling is ended.
8. The dispatching system for optimizing the cloud data center is characterized by comprising a monitoring module, a prediction module, a dispatching module, an evaluation module and a task classification module;
the monitoring module is used for monitoring the working condition of the physical machines and transmitting the cpu, ram and net resource condition of each physical machine to the scheduling module;
the prediction module is used for predicting the load condition of the virtual machine in the next period, namely, each resource requirement of the virtual machine will be reached in the next period, and transmitting the prediction information to the scheduling module;
the task classification module puts the new task into a corresponding task pool according to the classification;
the scheduling module acquires current resource duty ratio information of the physical machine from the monitoring module, and deploys tasks in the task pool according to the information;
the scheduling module also obtains prediction information of future load conditions of the virtual machine from the prediction module; generating a virtual machine scheduling plan according to the current resource duty ratio information and the prediction information of the physical machine; the scheduling module judges whether scheduling is carried out according to the virtual machine scheduling plan power consumption calculation result fed back by the evaluation module, and performs scheduling on the condition needing scheduling;
the evaluation module acquires a virtual machine scheduling plan generated by the scheduling module, and calculates power consumption generated by the virtual machine migration plan; and feeding back the calculation result to the scheduling module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011023815.3A CN112269632B (en) | 2020-09-25 | 2020-09-25 | Scheduling method and system for optimizing cloud data center |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011023815.3A CN112269632B (en) | 2020-09-25 | 2020-09-25 | Scheduling method and system for optimizing cloud data center |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112269632A CN112269632A (en) | 2021-01-26 |
| CN112269632B true CN112269632B (en) | 2024-02-23 |
Family
ID=74349404
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011023815.3A Active CN112269632B (en) | 2020-09-25 | 2020-09-25 | Scheduling method and system for optimizing cloud data center |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112269632B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113282366A (en) * | 2021-04-02 | 2021-08-20 | 中国船舶重工集团公司第七研究院 | Static virtual machine placement method oriented to embedded virtualization environment |
| CN113743728B (en) * | 2021-08-05 | 2022-11-01 | 北京航空航天大学 | Cluster detection equalization scheduling method considering state transition |
| CN113742090B (en) * | 2021-11-08 | 2022-03-08 | 苏州浪潮智能科技有限公司 | Load balancing method and device for virtual machine and storage medium |
| CN115237241B (en) * | 2022-09-26 | 2022-12-09 | 张北云联数据服务有限责任公司 | Energy-saving scheduling method and system for data center |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102096461A (en) * | 2011-01-13 | 2011-06-15 | 浙江大学 | Energy-saving method of cloud data center based on virtual machine migration and load perception integration |
| CN103576827A (en) * | 2012-07-25 | 2014-02-12 | 田文洪 | Method and device of online energy-saving dispatching in cloud computing data center |
| CN104199736A (en) * | 2014-06-30 | 2014-12-10 | 浙江大学苏州工业技术研究院 | Method for saving energy of data center under cloud environment |
| CN104657215A (en) * | 2013-11-19 | 2015-05-27 | 南京鼎盟科技有限公司 | Virtualization energy-saving system in Cloud computing |
| WO2015139374A1 (en) * | 2014-03-18 | 2015-09-24 | 成都盛思睿信息技术有限公司 | Virtual machine distributed task scheduling method in cloud computing platform |
| KR101578177B1 (en) * | 2014-09-12 | 2015-12-16 | 성균관대학교산학협력단 | Method and system for migration based on resource utilization rate in cloud computing |
| CN107861796A (en) * | 2017-11-30 | 2018-03-30 | 南京信息工程大学 | A kind of dispatching method of virtual machine for supporting cloud data center energy optimization |
| CN108089914A (en) * | 2018-01-18 | 2018-05-29 | 电子科技大学 | A kind of cloud computing deploying virtual machine algorithm based on energy consumption |
-
2020
- 2020-09-25 CN CN202011023815.3A patent/CN112269632B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102096461A (en) * | 2011-01-13 | 2011-06-15 | 浙江大学 | Energy-saving method of cloud data center based on virtual machine migration and load perception integration |
| CN103576827A (en) * | 2012-07-25 | 2014-02-12 | 田文洪 | Method and device of online energy-saving dispatching in cloud computing data center |
| CN104657215A (en) * | 2013-11-19 | 2015-05-27 | 南京鼎盟科技有限公司 | Virtualization energy-saving system in Cloud computing |
| WO2015139374A1 (en) * | 2014-03-18 | 2015-09-24 | 成都盛思睿信息技术有限公司 | Virtual machine distributed task scheduling method in cloud computing platform |
| CN104199736A (en) * | 2014-06-30 | 2014-12-10 | 浙江大学苏州工业技术研究院 | Method for saving energy of data center under cloud environment |
| KR101578177B1 (en) * | 2014-09-12 | 2015-12-16 | 성균관대학교산학협력단 | Method and system for migration based on resource utilization rate in cloud computing |
| CN107861796A (en) * | 2017-11-30 | 2018-03-30 | 南京信息工程大学 | A kind of dispatching method of virtual machine for supporting cloud data center energy optimization |
| CN108089914A (en) * | 2018-01-18 | 2018-05-29 | 电子科技大学 | A kind of cloud computing deploying virtual machine algorithm based on energy consumption |
Non-Patent Citations (1)
| Title |
|---|
| 云计算中基于负载预测的虚拟资源调度策略;王萍;付晓聪;许海洋;;青岛农业大学学报(自然科学版)(第01期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112269632A (en) | 2021-01-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112269632B (en) | Scheduling method and system for optimizing cloud data center | |
| US11579934B2 (en) | Scheduler for amp architecture with closed loop performance and thermal controller | |
| CN110413389A (en) | A kind of task schedule optimization method under the unbalanced Spark environment of resource | |
| CN104199739B (en) | A kind of speculating type Hadoop dispatching methods based on load balancing | |
| CN102622273A (en) | Self-learning load prediction based cluster on-demand starting method | |
| CN107533479B (en) | Power aware scheduling and power manager | |
| CN113672383A (en) | Cloud computing resource scheduling method, system, terminal and storage medium | |
| Fan | Job scheduling in high performance computing | |
| CN105740059B (en) | A kind of population dispatching method towards Divisible task | |
| Kao et al. | Data-locality-aware mapreduce real-time scheduling framework | |
| Zhang et al. | CloudFreq: Elastic energy-efficient bag-of-tasks scheduling in DVFS-enabled clouds | |
| KR101770736B1 (en) | Method for reducing power consumption of system software using query scheduling of application and apparatus for reducing power consumption using said method | |
| Kessaci et al. | Parallel evolutionary algorithms for energy aware scheduling | |
| Li et al. | Adaptive priority-based data placement and multi-task scheduling in geo-distributed cloud systems | |
| CN110221907A (en) | A kind of real-time task scheduling method based on EDF algorithm and fuzzy set | |
| Goubaa et al. | Scheduling periodic and aperiodic tasks with time, energy harvesting and precedence constraints on multi-core systems | |
| CN101685335A (en) | Application server based on SEDA as well as energy-saving device and method thereof | |
| KR101765830B1 (en) | Multi-core system and method for driving the same | |
| Zhu et al. | Energy-aware rolling-horizon scheduling for real-time tasks in virtualized cloud data centers | |
| Liu et al. | Leveraging dependency in scheduling and preemption for high throughput in data-parallel clusters | |
| Saroja et al. | Multi-criteria decision-making for heterogeneous multiprocessor scheduling | |
| Shah et al. | Phase level energy aware map reduce scheduling for big data applications | |
| Kim et al. | Cgroup-aware load balancing in heterogeneous multi-processor scheduler | |
| D'Souza et al. | Analysis of MapReduce scheduling and its improvements in cloud environment | |
| Zhu et al. | Multi-Objective Scheduling of Cloud Data Centers Prone to Failures. |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |