CN112269632A - Scheduling method and system for optimizing cloud data center - Google Patents

Scheduling method and system for optimizing cloud data center Download PDF

Info

Publication number
CN112269632A
CN112269632A CN202011023815.3A CN202011023815A CN112269632A CN 112269632 A CN112269632 A CN 112269632A CN 202011023815 A CN202011023815 A CN 202011023815A CN 112269632 A CN112269632 A CN 112269632A
Authority
CN
China
Prior art keywords
virtual machine
scheduling
machine
task
physical
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011023815.3A
Other languages
Chinese (zh)
Other versions
CN112269632B (en
Inventor
李云波
许乙付
张昌明
罗喜伶
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Innovation Research Institute of Beihang University
Original Assignee
Hangzhou Innovation Research Institute of Beihang University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Innovation Research Institute of Beihang University filed Critical Hangzhou Innovation Research Institute of Beihang University
Priority to CN202011023815.3A priority Critical patent/CN112269632B/en
Publication of CN112269632A publication Critical patent/CN112269632A/en
Application granted granted Critical
Publication of CN112269632B publication Critical patent/CN112269632B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/26—Power supply means, e.g. regulation thereof
    • G06F1/32—Means for saving power
    • G06F1/3203—Power management, i.e. event-based initiation of a power-saving mode
    • G06F1/3234—Power saving characterised by the action undertaken
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G06F9/5072—Grid computing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G06F9/5077—Logical partitioning of resources; Management or configuration of virtualized resources
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/104—Peer-to-peer [P2P] networks
    • H04L67/1044—Group management mechanisms 
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/45595—Network integration; Enabling network access in virtual machine instances
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/50—Indexing scheme relating to G06F9/50
    • G06F2209/5011—Pool
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The invention discloses a scheduling method and system for optimizing a cloud data center. Aiming at the problem that the resource scheduling mode of the existing cloud computing data center is low in efficiency, the scheduling method and the scheduling system comprise a streaming virtual machine deployment scheme and a periodic dynamic virtual machine scheduling scheme. The invention can monitor the resource utilization rate of the server and the virtual machine, and carry out virtual machine deployment by classifying the priority of different tasks. Under the condition of ensuring the QoS of a user, the periodic migration or the inactive non-migration of a small part of virtual machines can effectively improve the utilization rate of various resources of a server cluster and avoid unnecessary starting and closing (on-off) operations and corresponding energy loss of the server.

Description

Scheduling method and system for optimizing cloud data center
Technical Field
The invention belongs to the field of resource and energy consumption management of cloud data centers, and particularly relates to a method and a system for deployment and migration of virtual machines.
Background
In the last decade, energy consumption in data centers has attracted considerable attention in the field of information technology. Amazon america reported that over a three year cycle time, the energy consumption of its company's data center accounted for over 40% of the total economic cost. From the economic perspective, if the power consumption of the data center continues to increase at the present stage, the future power rate can easily exceed the hardware investment of the data center itself. At the same time, too high energy consumption increases the load of the power plant and power plants with non-clean energy may have further environmental impact.
The energy consumption of the data center is mainly distributed as follows: computing clusters, refrigeration systems, and others. The other accounts for 5% in total, and mainly comprises lamplight illumination and a UPS. The energy consumption of the refrigeration system accounts for 40 percent, and the energy consumption of the cluster is calculated to be 55 percent. At present, a part of novel small and medium-sized data centers greatly reduce the total power consumption of a refrigeration system by adopting a free cooling system, and can reduce the total power consumption to half of the total power consumption all the year round at most. Therefore, the maximum power consumption of the computing cluster can reach 75% of the overall power consumption of the data center. In order to reduce the power consumption of the data center, the technology for optimizing the energy consumption of the computer group is very critical. The energy consumption of the computing cluster is mainly divided into static energy consumption and dynamic energy consumption. The static energy consumption means that the basic power consumption of a single machine of a physical machine accounts for 40% of the total power consumption of the single machine when the physical machine is not in a standby state for executing specific tasks; the dynamic energy consumption refers to the power consumption increased on the basis of static energy consumption when a physical machine executes tasks, and the dynamic energy consumption mainly comprises a CPU (central processing unit) and an RAM (random access memory), and the energy consumption of a network card and a hard disk is relatively low.
In cloud computing, virtualization is a common resource management technique. Virtualization abstracts various entities of a computer, such as a CPU, RAM, and a hard disk. In the running task, different tasks run on different virtual machines, and the different virtual machines can be selected to run on the same or different physical machines.
The cloud computing data center cluster is virtualized, when a task arrives, a system can create a virtual machine with a corresponding size, and the task can be completed in the virtual machine.
Regarding resource management, the virtual machine deployment mostly adopts offline methods such as genetic algorithm, simulated annealing and the like, the optimization goal is load balancing among physical machines, and a single machine has a higher load rate as much as possible.
With respect to power consumption, the mainstream solution today is to migrate and merge virtual machines, sleep or shut down portions of physical machines to reduce power consumption.
The main problems of the prior art solutions are the following,
1. the control module often migrates and merges the virtual machines according to the current resource utilization rate. After the migration merge action is completed, the current server may be put to sleep or powered off. However, often just after the server is shut down, a new task arrives soon and the server needs to be reactivated. In the whole process of turning on and off, the migration, system suspension and starting of the virtual machine occupy CPU resources and time, and more energy consumption is caused. In other words, the impact of the timing of decision merging on energy consumption is also large.
2. Most of the existing task scheduling algorithms are mostly offline in order to ensure higher cluster resource utilization rate. At the beginning of the scheduling algorithm, all tasks to be run are submitted to the scheduling module. And the scheduling module creates a virtual machine according to the size of the resource required by the task and allocates the virtual machine to a physical machine with enough resources in the cluster for running. After a new task arrives, in order to enable the cluster resources to achieve a better utilization rate, all the running virtual machines need to be redeployed, and the redeployment of all the virtual machines is costly and may affect the user service quality of part of the tasks. For example, the existing live migration technology for the virtual machine may cause a short time interruption of tasks in the virtual machine when the virtual machine is migrated, and particularly may cause a great influence on some tasks providing web-services.
3. The current mainstream online algorithm mainly uses the utilization rate of the cpu as a scheduling reference unit, because the cpu is the most energy-consuming component in a physical stand-alone machine. If only the cpu is used as the basic scheduling unit, the cpu occupancy rate of a part of physical machines is high, the utilization rate of the memory or the network is low, and the internal resource usage is very unbalanced, which causes resource waste. For example, there are machine cpu resources that are still surplus, but the memory and network are full. The server has three physical resources of CPU, ram and net, and a reasonable deployment mode should be placed by comprehensive consideration of CPU, memory and network resource multivariable.
4. The offline virtual machine deployment algorithm requires a longer computation time than the online algorithm, and a deployment scenario requiring recalculation of all virtual machines is reached every time a new task arrives.
Disclosure of Invention
Aiming at the problem that the resource scheduling mode of the existing cloud computing data center is low in efficiency, the invention provides an online virtual machine deployment scheme for optimizing resources and energy consumption of the cloud computing data center and a periodic dynamic virtual machine scheduling converging method. The invention can monitor the resource utilization rates of the server and the virtual machines, periodically and quickly adjust the previous deployment scheme of the virtual machines while ensuring the service quality of users by classifying the priorities of different tasks, and improve the resource utilization rate of the local server so as to reduce the total power consumption.
The invention firstly discloses a scheduling method for optimizing cloud data center resources, which comprises a streaming virtual machine deployment step and a periodic dynamic virtual machine scheduling step, wherein a newly arrived task is deployed according to the streaming virtual machine deployment step; in each set scheduling period, judging whether to perform virtual machine scheduling according to the periodic dynamic virtual machine scheduling step, and executing according to the judgment result;
the step of deploying the streaming virtual machine comprises the following steps:
1) when the task arrives, classifying the task according to the task type, and putting the task into a corresponding task pool to wait for scheduling; the task types comprise a time delay sensitive type and a non-time delay sensitive type;
2) acquiring the cpu, ram and net resource conditions of each current physical machine, and summarizing the ratio of each resource left by each physical machine in an on state; performing simulated placement, namely after the current virtual machine is placed into each physical machine with the on state, summarizing the remaining resource proportion of each physical machine; selecting the candidate machine with the optimal resource occupation ratio to place a new task for the first candidate machine; if no candidate machine exists, selecting any physical machine with the off state to place a new task or returning the task to the task pool to continue waiting according to the task type;
the scheduling step of the periodic dynamic virtual machine comprises the following steps:
A) acquiring the load condition of each physical machine, acquiring the sum of all resource requirements of the virtual machines to be reached in the next period, generating a virtual machine migration plan,
B) judging whether to execute the virtual machine migration plan in the scheduling period; if so, migrating the virtual machine according to the virtual machine migration plan; otherwise, the scheduling of the scheduling period is finished.
The invention also discloses a scheduling system for optimizing the cloud data center resources, which comprises a monitoring module, a prediction module, a scheduling module, an evaluation module and a task classification module;
the task classification module puts the new tasks into corresponding task pools according to the categories;
the monitoring module is used for monitoring the working condition of the physical machine and transmitting the cpu, ram and net resource condition of each physical machine to the scheduling module;
the prediction module is used for predicting the load condition of the virtual machine in the next period, namely, each resource demand of the virtual machine in the next period is reached, and the prediction information is transmitted to the scheduling module;
the scheduling module acquires the current resource proportion information of the physical machine from the monitoring module and carries out the tasks in the task pool on the physical machine;
the scheduling module also acquires prediction information of the future load condition of the virtual machine from the prediction module; generating a virtual machine scheduling plan according to the current resource proportion information and the prediction information of the physical machine; the scheduling module judges whether to perform scheduling according to the power consumption calculation result of the virtual machine scheduling plan fed back by the evaluation module, and performs scheduling on the condition needing to be scheduled;
the evaluation module acquires a virtual machine scheduling plan generated by the scheduling module and calculates the power consumption generated by the virtual machine migration plan; and feeding back the calculation result to the scheduling module.
The invention can provide a dynamic resource management method for a cloud data center. The periodic migration or inactive non-migration of a small part of virtual machines can effectively improve the utilization rate of various resources of the server cluster, and avoid unnecessary starting and closing (on-off) operations and corresponding energy loss of the server.
Drawings
FIG. 1 is a diagram of a physical machine (including the ratio of resources left in each item);
fig. 2 is a schematic diagram of an optimized cloud data center scheduling system according to the present invention.
Detailed Description
The invention will be further illustrated and described with reference to specific embodiments. The technical features of the embodiments of the present invention can be combined correspondingly without mutual conflict. In the present invention, unless otherwise specified, the physical machine and the server are equivalent in concept and property.
The method comprises a streaming virtual machine deployment step and a periodic dynamic virtual machine scheduling step, wherein a new task is deployed according to the streaming virtual machine deployment step; and in each set scheduling period, judging whether to carry out virtual machine scheduling according to the periodic dynamic virtual machine scheduling step, and executing according to the judgment result.
Task vmiExpressed as: vmi＝(si，ei，di，type，<cpu，ram，net>，mid)
Wherein s isiRefers to the start time of this task; e.g. of the typeiRefers to the task ending time; diIndicating the estimated time of the task; type refers to task type, and is b (batch-service, i.e. non-delay sensitive task) or w (web-service, i.e. delay sensitive task);<cpu，ram，net>respectively corresponding to the resources required by the virtual machine where the task is located: cpu refers to the number of cpu cores required by the virtual machine, ram refers to the memory capacity required by the virtual machine, and net refers to the network bandwidth required by the virtual machine; m isidRepresenting the physical machine number where the virtual machine is located.
The delay sensitive task refers to a type of task which is not interrupted as much as possible in the running process;
the non-delay sensitive task can be interrupted in the running process, then stored as a snapshot (snapshot), and then can be waken to restore the running process.
As shown in FIG. 1, the resources of each physical machine are divided into three<cpu，ram，net>The cpu is the number of the cpus of the physical machines, ram is the memory capacity, and net is the network bandwidth of the physical machines. The physical machine has two states: on and off. on means the current physical machine is in an active state (high power consumption) and off means the current physical machine is in a sleep or off state (low or no power consumption). The two states can be switched with each other, and the energy consumption lost in the switching can be respectively described as switching from an operation (on) to a sleep (off) Switchoff＝toff*eoff，toffRefers to the time required to switch from running to sleep; e.g. of the typeoffThe unit time power consumption of the physical machine is switched from the running state to the sleeping state; switch from sleep (off) to sleep (on) Switchon＝ton*eon，tonThe time required for switching from sleep to operation, eonRefers to the power consumption per unit time of the physical machine switching from sleep to run state.
On-line virtual machine deployment scheme for cloud computing data center
The task is submitted to the cloud by the user, and the task is also submitted with the corresponding requirements of the virtual machine resources. The scheduling module obtains resource indexes required by the virtual machine, wherein the resource indexes comprise a cpu, an ram and a net. Cpu refers to the number of Cpu cores required by the virtual machine; ram refers to the memory capacity required by the virtual machine; net refers to the network bandwidth required by a virtual machine. According to the task types submitted by users, the tasks are classified into delay sensitive types and non-delay sensitive types. Latency sensitive is generally referred to as web-service like tasks, such as shopping websites, clothing websites, personal blogs, etc. Such tasks are typically long and very sensitive to latency, and migrating such virtual machines may result in a brief service suspension, so such tasks are not suitable for migration. On the contrary, the non-delay-sensitive multi-finger computing task (batch-service) is relatively insensitive to delay, and the task is mostly read-write intensive (I/O-intensive) or compute-intensive (cpu-intensive). Such tasks typically have a specific computation deadline (deadline), i.e., they are completed within a specified time.
And the scheduling module collects the resource occupation ratios of all the resources left by each physical machine after acquiring the current cpu, ram and net resource conditions of each physical machine from the monitoring module.
And then, the scheduling module performs simulated placement, namely after the current virtual machine is placed into each physical machine, the remaining resource proportion of each item of each physical machine is summarized. And selecting the machine with the optimal resource ratio as a first candidate machine. After task placement, the same calculation is performed for the next arriving task.
Two, periodic dynamic virtual machine scheduling confluence
The control module obtains load information of each physical machine and load information of the virtual machines from the monitoring module at the beginning of each time period (e.g., one hour). Meanwhile, the scheduling module acquires the prediction information of the future load condition of the virtual machine from the prediction module, and the scheduling module determines whether to schedule the virtual machine in the time period according to the information.
If the scheduling is improper or too frequent, unnecessary virtual machine migration overhead may cause power consumption of the entire cluster to increase. The scheduling module should be as inert as possible to determine whether to schedule in the current cycle. If the dispatching is needed, the dispatching module generates a dispatching plan, then the appointed virtual machine is migrated to the target physical machine, and then the partial physical machine is closed or sleeped, so that the resource utilization rate of the cluster is improved, and the overall power consumption of the cluster is reduced.
And traversing all the non-delay sensitive tasks by the scheduling module, selecting part of the virtual machines meeting the conditions, and adding the virtual machines into the migration plan. For the tasks of the delay sensitive type, the tasks are very sensitive to the delay, and the service interruption can be caused when the virtual machine is migrated, so that the scheduling module does not preferentially perform the migration operation on the virtual machine of the tasks.
And after the scheduling module generates a part of virtual machine migration plan of the time interval, calculating the power consumption generated by the scheduling. The virtual machine migration cost function comprises: the power consumption of cpu and ram consumed by virtual machine migration, and the power consumption of a network in the virtual machine migration process. And after the migration of the virtual machine group is finished, the server enters dormant power consumption and awakening power consumption. And (4) migrating a reward function of the virtual machine, namely the power consumption which can be reduced after migration. The migration is too frequent, which causes a large network and energy consumption overhead, so the scheduling should be performed according to the principle of lazy scheduling, that is, only a virtual machine migration plan which can bring about gains is executed.
If the reward obtained after migration is less than the cost of migration, the virtual machine migration plan with excessive power consumption will not be executed. Instead, the virtual machine migration plan through the evaluation will be executed immediately. And after the virtual machine is migrated, sleeping or closing the appointed server. The periodic scheduling is finished.
The main optimization target of the scheduling algorithm is to improve the comprehensive resource utilization rate of the physical machine by sleeping or turning off the unnecessary physical machine. .
As shown in fig. 2, the following describes the specific process of the dynamic resource scheduling method of the present invention in detail based on the legend.
And when the task arrives, the task is respectively put into the corresponding task pools according to the type of the task to wait for scheduling. At the moment, the scheduling module acquires the remaining resource occupation condition u of each physical machine from the physical machine monitoring modulecpu，uram，unet。
Wherein
Figure BDA0002701537140000061
Representing the resources left by the current cpu;
Figure BDA0002701537140000062
representing the remaining memory resources of the current physical machine;
Figure BDA0002701537140000063
representing the network resources left by the current physical machine;
for all statesFor each physical machine on, the calculation is performed:
Figure BDA0002701537140000064
Figure BDA0002701537140000065
alpha, beta, gamma are weight coefficients, vcpuCpu resources (e.g., cpu core number), v, occupied for a virtual machineramMemory resources (e.g., memory size), v, occupied for a virtual machinenetIs a network resource of the virtual machine. After the calculation is completed, all the physical machines are processed according to pm _ scoreiValue of (pm _ score)i> 0) generates a candidate server list sorted from large to small. And if the task is a web-service class task, directly placing the virtual machine into the physical machine ranked on the candidate server list 1. If the candidate list is empty, any physical machine with the state of off is selected and the current virtual machine is placed into the physical machine.
If the task is a non-delay sensitive task, and no one of the current physical machines in the on state has enough resources to deploy the virtual machine, computing a slice, where the slice is e, of the task in the virtual machinei-si-di. If the sleep is more than or equal to 0, the task is put into a non-delay sensitive task pool to continue waiting; if the slack is less than 0, the task is promoted in priority and treated by a delay sensitive task.
Each time a new batch-service class task is returned to the batch pool, the scheduling module performs the following calculations
Figure BDA0002701537140000076
When in use
Figure BDA0002701537140000072
When the total demand of the task cpus in the batch pool exceeds 50% of the cpu resource of a single physical machine, selecting any physical machine with an off state to start, and placing the virtual machines corresponding to the tasks in all the batch pools on the physical machine.
When the scheduling module detects that the slack of a task in the batch pool is less than 1, selecting any physical machine with the status of off to start, and placing the virtual machines corresponding to the tasks in all the batch pools on the physical machine.
The periodic virtual machine migration plan is as follows:
a latency sensitive task (web-service) may be described as < cpu _ use, ram _ use, network _ use, c _ mig, duration >. Three parameters of cpu _ usage, ram _ usage and network _ usage in the delay-sensitive task refer to the utilization rate of cpu, the utilization rate of memory and the utilization rate of network bandwidth in the current time period, and the cpu _ usage, the ram _ usage and the net _ usage are variables; migration cost
Figure BDA0002701537140000073
τ is usually a constant greater than 1, and should not migrate web-service type tasks as much as possible so as not to affect user quality of service; e.g. of the typemThe power consumption of a physical machine and a network in unit time during the migration of the virtual machine is calculated;
Figure BDA0002701537140000074
time required to migrate a virtual machine; c. CmigIs a migration cost; the duration (duration) is typically a constant.
A non-latency sensitive task (batch-service) may be described as<cpu_usage，ram_usage，network_usage，cmig，duration，deadline>. CPU _ use, ram _ use and network in the Batch-service task are usually small-variation quantities and can be regarded as invariant. Migration cost
Figure BDA0002701537140000075
Compared to web-service, cmigWithout the coefficient tau, a smaller migration cost can be obtained under the same condition, and the priority of the migration non-delay sensitive type is prior to the delay sensitive type; the duration of the non-delay sensitive type is typically a finite time.
At the beginning of each time period, the scheduling module obtains current resource proportion information of the physical machine from the monitoring module. Counting the total of all the current cpus, the memories and the network resources of all the physical machines with the on states:
Figure BDA0002701537140000081
Figure BDA0002701537140000082
the scheduling module obtains resource requirements of a cpu, a memory and a network which will reach the virtual machine in the next period from the prediction module, and the scheduling module obtains the sum of the resource requirements of the virtual machine in the next period from the prediction module
Figure BDA0002701537140000083
Respectively calculate r1-r′1，r2-r′2，r3-r′3If one item is less than 0, the virtual machine is not migrated in the period. If r is1-r′1，r2-r′2，r3-r′3When the number of the scheduling plans is more than or equal to 0, the scheduling module starts to generate a scheduling plan:
for each physical machine i with all states on, a calculation is performed:
Figure BDA0002701537140000084
Figure BDA0002701537140000085
Figure BDA0002701537140000086
and generating three corresponding lists, namely a list _ machine _ CPU, a list _ machine _ RAM and a list _ machine _ NET according to the obtained scores from large to small. Finally, the score is calculated for each physical machine:
Figure BDA0002701537140000087
the obtained pm _ scoreiAnd generating a list _ machine of the physical machines according to the descending order.
Physical machine pm1Rank first in list _ machine, so move it into the migration manifest, pm1Based on the virtual machine scores:
Figure BDA0002701537140000088
sequencing is carried out, and the pm _ CPU _ score corresponds to the item with the highest CPU/ram/net in the current virtual machinei，pm_RAM_scorei，pm_NET_scoreiAnd matching the physical machine before the migration. For example, vm _ scoreiThe highest score in the score is
Figure BDA0002701537140000089
Then at pm _ CPU _ scoreiAnd selecting the physical machines meeting the resource requirements of the virtual machines as migration objects according to the sequence of the list. And so on until the physical machine pm1All virtual machines on are matched to the target migration server. And the scheduling module adds the migration scheme of the virtual machine in the physical machine into the total scheduling plan of the current time period. Then all the physical machines pm _ score in the list _ machine are updated againi. According to the method, the physical machine ranked at 1 in the current list-machine tries to generate a virtual machine migration scheme until all the virtual machines in a certain physical machine cannot be migrated into all the current physical machines with the on states, and the scheduling module terminates the calculation.
At this time, the scheduling module calculates the power consumption generated by the scheduling. The virtual machine migration cost function comprises: the power consumption of cpu and ram consumed by virtual machine migration, and the power consumption of a network in the virtual machine migration process. And after the migration of the virtual machine group is finished, the server enters dormant power consumption and awakening power consumption. And (4) migrating a reward function of the virtual machine, namely the power consumption which can be reduced after migration. The migration is too frequent, which causes a large network and energy consumption overhead, so the scheduling should be performed according to the principle of lazy scheduling, that is, only a virtual machine migration plan which can bring about gains is executed.
If the reward obtained after migration is less than the cost of migration, the virtual machine migration plan with excessive power consumption will not be executed. Instead, the virtual machine migration plan through the evaluation will be executed immediately.
The virtual machine migration reward function is defined as:
Figure BDA0002701537140000091
wherein i corresponds to all virtual machines needing to be migrated in the migration plan, and i corresponds to physical machines needing to be shut down.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims (8)

1. A scheduling method for optimizing a cloud data center is characterized by comprising the following steps:
the method comprises a streaming virtual machine deployment step and a periodic dynamic virtual machine scheduling step, wherein a new task is deployed according to the streaming virtual machine deployment step; in each set scheduling period, judging whether to perform virtual machine scheduling according to the periodic dynamic virtual machine scheduling step, and executing according to the judgment result;
the step of deploying the streaming virtual machine comprises the following steps:
1) when the task arrives, classifying the task according to the task type, and putting the task into a corresponding task pool to wait for scheduling; the task types comprise a time delay sensitive type and a non-time delay sensitive type;
2) acquiring the cpu, ram and net resource conditions of each current physical machine, and summarizing the ratio of each resource left by each physical machine in an on state; performing simulated placement, namely after the current virtual machine is placed into each physical machine with the on state, summarizing the remaining resource proportion of each physical machine; selecting a physical machine with the optimal resource ratio as a first candidate machine to place a new task; if no candidate machine exists, selecting any physical machine with the off state to place a new task or returning the task to the task pool to continue waiting according to the task type;
the scheduling step of the periodic dynamic virtual machine comprises the following steps:
A) acquiring the load condition of each physical machine, acquiring the sum of all resource requirements which will reach the virtual machine in the next period, and judging whether to generate a virtual machine migration plan, wherein the virtual machine migration plan is generated only when the sum of all the current cpu, the memory and the network resources of all the physical machines in the on state is not less than the sum of the resource requirements which will reach the virtual machine in the next period;
B) judging whether to execute the virtual machine migration plan in the scheduling period; if so, migrating the virtual machine according to the virtual machine migration plan; otherwise, the scheduling of the scheduling period is finished.
2. The scheduling method for optimizing the cloud data center according to claim 1, wherein the task in the step 1) is recorded as vmiIt is expressed as: vmi＝(si，ei，di，type，<cpu，ram，net>，mid)
Wherein s isiRefers to the start time of this task; e.g. of the typeiRefers to the task ending time; diIndicating the estimated time of the task; type refers to task type;<cpu，ram，net>respectively corresponding to the resources required by the virtual machine where the task is located: cpu refers to the number of cpu cores required by the virtual machine, ram refers to the memory capacity required by the virtual machine, and net refers to the network bandwidth required by the virtual machine; m isidRepresenting the physical machine number where the virtual machine is located.
3. The scheduling method for optimizing the cloud data center according to claim 1 or 2, wherein the step 2) is specifically as follows:
acquiring the remaining resource occupation conditions of all items of each physical machine with an on state: u. ofcpu，uram，unet，
Wherein
Figure FDA0002701537130000021
Representing the resources left by the current cpu;
Figure FDA0002701537130000022
representing the remaining memory resources of the current physical machine;
Figure FDA0002701537130000023
representing the network resources left by the current physical machine;
for each physical machine i with all states on, a calculation is performed:
Figure FDA0002701537130000024
Figure FDA0002701537130000025
alpha, beta, gamma are weight coefficients, vcpuCpu resources, v, occupied for virtual machinesramMemory resources, v, occupied by virtual machinesnetIs the network resource of the virtual machine, and is calculated according to pm _ scoreiThe value of (pm) is used for sorting all physical machines from large to small to generate a candidate server list, wherein pm _ scorei＞0；
For the time delay sensitive task, directly placing the virtual machine into the physical machine with the rank 1 in the candidate server list; if the candidate list is empty, selecting any physical machine with the state of off and placing the current virtual machine into the physical machine;
for a non-delay sensitive task, calculating a time tolerance slack for the task, which is specifically defined as: slice ═ ei-si-di(ii) a If the sleep is more than or equal to 0, the task is put into a non-delay sensitive task pool to continue waiting; if the slack is less than 0, the task is promoted in priority and treated by a delay sensitive task.
4. The scheduling method for optimizing the cloud data center according to claim 3, wherein the following calculation is performed each time when a new non-delay-sensitive task is returned to the non-delay-sensitive task pool
Figure FDA0002701537130000026
When in use
Figure FDA0002701537130000027
When the virtual machine is started, selecting any one off physical machine to start, and placing the virtual machines corresponding to the tasks in the non-delay sensitive task pool on the physical machine; wherein m iscpuThe method is characterized in that the method is a single physical machine cpu resource;
when detecting that the slack of a task in the non-delay sensitive task pool is less than 1, selecting any physical machine with the state of off to start, and placing the virtual machines corresponding to the tasks in the batch pool on the physical machine.
5. The scheduling method for optimizing the cloud data center according to claim 1, wherein the step a) is specifically as follows:
at the beginning of each time period, the scheduling module acquires the current resource proportion information of the physical machine from the monitoring module, and counts the sum r of all the resources of the current cpu, the memory and the network of all the physical machines in the on state1，r2，r3The scheduling module obtains the cpu, the memory and each resource requirement r 'of the network to the virtual machine in the next period from the prediction module'1，r′2，r′3Separately calculate r1-r′1，r2-r′2，r3-r′3If one item is smaller than 0, the scheduling plan is not generated in the period, namely, the virtual machine migration is not carried out; if r is1-r′1，r2-r′2，r3-r′3When the number of the scheduling plans is more than or equal to 0, the scheduling module starts to generate the scheduling plan.
6. The scheduling method for optimizing the cloud data center according to claim 1 or 5, wherein the generating of the scheduling plan specifically includes:
for each physical machine i with all states on, a calculation is made:
Figure FDA0002701537130000031
Figure FDA0002701537130000032
Figure FDA0002701537130000033
according to the obtained scores, three corresponding lists, namely a list _ machine _ CPU, a list _ machine _ RAM and a list _ machine _ NET, are generated from large to small;
finally, the score is calculated for each physical machine:
Figure FDA0002701537130000034
Figure FDA0002701537130000035
the obtained pm _ scoreiGenerating a list _ machine of the physical machine according to the sequence from big to small;
physical machine pm1Rank first in list _ machine, move it into migration manifest, pm1Based on the virtual machine scores:
Figure FDA0002701537130000036
sequencing is carried out, and the pm _ CPU _ score corresponds to the item with the highest CPU/ram/net in the current virtual machinei，pm_RAM_scorei，pm_NET_scoreiMatching the physical machines before migration; until the physical machine pm1All the virtual machines are matched with a target migration server;
the scheduling module adds the migration scheme of the virtual machines in the physical machines into the total scheduling plan of the current time period, and then renews and updates all the physical machines pm _ score in the list _ machinei(ii) a And then generating a virtual machine migration scheme for the physical machines ranked at 1 st in the current list-machine until all the virtual machines in a certain physical machine cannot be migrated to all the current physical machines in the on state, and terminating the calculation of the scheduling module to obtain a final scheduling plan.
7. The scheduling method for optimizing the cloud data center according to claim 1, wherein the step B) is specifically as follows:
calculating the power consumption generated by the virtual machine migration plan, namely calculating a virtual machine migration cost function, wherein the virtual machine migration cost function comprises the power consumption of cpu and ram consumed by virtual machine migration, the power consumption of a network in the virtual machine migration process, and the power consumption of the physical machine entering dormancy and the power consumption of awakening after the virtual machine cluster migration is finished; calculating a virtual machine migration reward function, namely power consumption which can be reduced after migration;
if the reward obtained after migration is less than the cost of migration, the virtual machine migration plan will not be executed; on the contrary, the evaluated virtual machine migration plan is executed immediately, after the virtual machine migration is finished, the designated physical machine is put to sleep or shut down, and the cycle scheduling is finished.
8. The scheduling system for optimizing the cloud data center is characterized by comprising a monitoring module, a prediction module, a scheduling module, an evaluation module and a task classification module;
the monitoring module is used for monitoring the working condition of the physical machine and transmitting the cpu, ram and net resource condition of each physical machine to the scheduling module;
the prediction module is used for predicting the load condition of the virtual machine in the next period, namely, each resource demand of the virtual machine in the next period is reached, and the prediction information is transmitted to the scheduling module;
the task classification module puts the new tasks into corresponding task pools according to the categories;
the scheduling module acquires the current resource proportion information of the physical machine from the monitoring module and deploys tasks in the task pool according to the information;
the scheduling module also acquires prediction information of the future load condition of the virtual machine from the prediction module; generating a virtual machine scheduling plan according to the current resource proportion information and the prediction information of the physical machine; the scheduling module judges whether to perform scheduling according to the power consumption calculation result of the virtual machine scheduling plan fed back by the evaluation module, and performs scheduling on the condition needing to be scheduled;
the evaluation module acquires a virtual machine scheduling plan generated by the scheduling module and calculates the power consumption generated by the virtual machine migration plan; and feeding back the calculation result to the scheduling module.
CN202011023815.3A 2020-09-25 2020-09-25 Scheduling method and system for optimizing cloud data center Active CN112269632B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011023815.3A CN112269632B (en) 2020-09-25 2020-09-25 Scheduling method and system for optimizing cloud data center

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011023815.3A CN112269632B (en) 2020-09-25 2020-09-25 Scheduling method and system for optimizing cloud data center

Publications (2)

Publication Number Publication Date
CN112269632A true CN112269632A (en) 2021-01-26
CN112269632B CN112269632B (en) 2024-02-23

Family

ID=74349404

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011023815.3A Active CN112269632B (en) 2020-09-25 2020-09-25 Scheduling method and system for optimizing cloud data center

Country Status (1)

Country Link
CN (1) CN112269632B (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113282366A (en) * 2021-04-02 2021-08-20 中国船舶重工集团公司第七研究院 Static virtual machine placement method oriented to embedded virtualization environment
CN113742090A (en) * 2021-11-08 2021-12-03 苏州浪潮智能科技有限公司 Load balancing method and device for virtual machine and storage medium
CN113743728A (en) * 2021-08-05 2021-12-03 北京航空航天大学 Cluster detection equalization scheduling method considering state transition
CN115237241A (en) * 2022-09-26 2022-10-25 张北云联数据服务有限责任公司 Energy-saving scheduling method and system for data center

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102096461A (en) * 2011-01-13 2011-06-15 浙江大学 Energy-saving method of cloud data center based on virtual machine migration and load perception integration
CN103576827A (en) * 2012-07-25 2014-02-12 田文洪 Method and device of online energy-saving dispatching in cloud computing data center
CN104199736A (en) * 2014-06-30 2014-12-10 浙江大学苏州工业技术研究院 Method for saving energy of data center under cloud environment
CN104657215A (en) * 2013-11-19 2015-05-27 南京鼎盟科技有限公司 Virtualization energy-saving system in Cloud computing
WO2015139374A1 (en) * 2014-03-18 2015-09-24 成都盛思睿信息技术有限公司 Virtual machine distributed task scheduling method in cloud computing platform
KR101578177B1 (en) * 2014-09-12 2015-12-16 성균관대학교산학협력단 Method and system for migration based on resource utilization rate in cloud computing
CN107861796A (en) * 2017-11-30 2018-03-30 南京信息工程大学 A kind of dispatching method of virtual machine for supporting cloud data center energy optimization
CN108089914A (en) * 2018-01-18 2018-05-29 电子科技大学 A kind of cloud computing deploying virtual machine algorithm based on energy consumption

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102096461A (en) * 2011-01-13 2011-06-15 浙江大学 Energy-saving method of cloud data center based on virtual machine migration and load perception integration
CN103576827A (en) * 2012-07-25 2014-02-12 田文洪 Method and device of online energy-saving dispatching in cloud computing data center
CN104657215A (en) * 2013-11-19 2015-05-27 南京鼎盟科技有限公司 Virtualization energy-saving system in Cloud computing
WO2015139374A1 (en) * 2014-03-18 2015-09-24 成都盛思睿信息技术有限公司 Virtual machine distributed task scheduling method in cloud computing platform
CN104199736A (en) * 2014-06-30 2014-12-10 浙江大学苏州工业技术研究院 Method for saving energy of data center under cloud environment
KR101578177B1 (en) * 2014-09-12 2015-12-16 성균관대학교산학협력단 Method and system for migration based on resource utilization rate in cloud computing
CN107861796A (en) * 2017-11-30 2018-03-30 南京信息工程大学 A kind of dispatching method of virtual machine for supporting cloud data center energy optimization
CN108089914A (en) * 2018-01-18 2018-05-29 电子科技大学 A kind of cloud computing deploying virtual machine algorithm based on energy consumption

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
王萍;付晓聪;许海洋;: "云计算中基于负载预测的虚拟资源调度策略", 青岛农业大学学报(自然科学版), no. 01 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113282366A (en) * 2021-04-02 2021-08-20 中国船舶重工集团公司第七研究院 Static virtual machine placement method oriented to embedded virtualization environment
CN113743728A (en) * 2021-08-05 2021-12-03 北京航空航天大学 Cluster detection equalization scheduling method considering state transition
CN113742090A (en) * 2021-11-08 2021-12-03 苏州浪潮智能科技有限公司 Load balancing method and device for virtual machine and storage medium
CN113742090B (en) * 2021-11-08 2022-03-08 苏州浪潮智能科技有限公司 Load balancing method and device for virtual machine and storage medium
CN115237241A (en) * 2022-09-26 2022-10-25 张北云联数据服务有限责任公司 Energy-saving scheduling method and system for data center
CN115237241B (en) * 2022-09-26 2022-12-09 张北云联数据服务有限责任公司 Energy-saving scheduling method and system for data center

Also Published As

Publication number Publication date
CN112269632B (en) 2024-02-23

Similar Documents

Publication Publication Date Title
CN112269632B (en) Scheduling method and system for optimizing cloud data center
US11720403B2 (en) System for commitment-aware workload scheduling based on anticipated resource consumption levels
Sharma et al. Performance analysis of load balancing algorithms
Hui et al. Improved strategies for dynamic load balancing
CN103729248B (en) A kind of method and apparatus of determination based on cache perception task to be migrated
Rajguru et al. A comparative performance analysis of load balancing algorithms in distributed system using qualitative parameters
CN102662763B (en) Virtual machine resource scheduling method based on service quality
CN109766180B (en) Load balancing method and device, storage medium, computing equipment and computing system
CN110231976B (en) Load prediction-based edge computing platform container deployment method and system
CN104199739B (en) A kind of speculating type Hadoop dispatching methods based on load balancing
CN110413389A (en) A kind of task schedule optimization method under the unbalanced Spark environment of resource
Wang et al. Fresh: Fair and efficient slot configuration and scheduling for hadoop clusters
US10089155B2 (en) Power aware work stealing
CN105740059B (en) A kind of population dispatching method towards Divisible task
CN101582043A (en) Dynamic task allocation method of heterogeneous computing system
Kao et al. Data-locality-aware mapreduce real-time scheduling framework
Fan Job scheduling in high performance computing
CN108446179A (en) Priority weights Task method for scheduling task based on load-balancing mechanism
CN104917839A (en) Load balancing method for use in cloud computing environment
Li et al. Adaptive priority-based data placement and multi-task scheduling in geo-distributed cloud systems
Emara et al. Genetic-Based Multi-objective Task Scheduling Algorithm in Cloud Computing Environment.
Liu et al. Leveraging dependency in scheduling and preemption for high throughput in data-parallel clusters
Saroja et al. Multi-criteria decision-making for heterogeneous multiprocessor scheduling
Das et al. Mapreduce scheduler: A 360-degree view
D'Souza et al. Analysis of MapReduce scheduling and its improvements in cloud environment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant