CN110908772A - Energy-saving scheduling method for improving reliability of multiple workflows - Google Patents

Energy-saving scheduling method for improving reliability of multiple workflows Download PDF

Info

Publication number
CN110908772A
CN110908772A CN201911113879.XA CN201911113879A CN110908772A CN 110908772 A CN110908772 A CN 110908772A CN 201911113879 A CN201911113879 A CN 201911113879A CN 110908772 A CN110908772 A CN 110908772A
Authority
CN
China
Prior art keywords
task
virtual machine
reliability
curr
tasks
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201911113879.XA
Other languages
Chinese (zh)
Other versions
CN110908772B (en
Inventor
夏元清
陈岳红
戴荔
张金会
翟弟华
刘坤
闫莉萍
邹伟东
孙中奇
郭泽华
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Institute of Technology BIT
Original Assignee
Beijing Institute of Technology BIT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Institute of Technology BIT filed Critical Beijing Institute of Technology BIT
Priority to CN201911113879.XA priority Critical patent/CN110908772B/en
Publication of CN110908772A publication Critical patent/CN110908772A/en
Application granted granted Critical
Publication of CN110908772B publication Critical patent/CN110908772B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • G06F9/4893—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues taking into account power or heat criteria
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/48—Indexing scheme relating to G06F9/48
    • G06F2209/484—Precedence
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The invention provides an energy-saving scheduling method for improving the reliability of multiple workflows, which improves the reliability of workflows submitted by multiple users and simultaneously reduces the energy consumption of a data center by using a virtual machine frequency modulation technology through two stages of workflow pre-scheduling and task re-scheduling based on the reliability and deadline requirements of the users. The invention relates to a multi-workflow energy-saving scheduling algorithm based on two stages of pre-scheduling and re-scheduling, wherein workflow task sequencing and workflow reliability constraint decomposition are optimized in a workflow pre-scheduling stage; and in the task rescheduling stage, according to the task reliability target value obtained in the pre-scheduling stage, a virtual machine frequency modulation technology is used for reselecting the virtual machine which meets the reliability target value and has the minimum energy consumption for the task, and the mapping relation between the optimized task and the virtual machine is obtained, so that the reliability of the multi-workflow is improved, and the operation energy consumption of the data center is reduced.

Description

Energy-saving scheduling method for improving reliability of multiple workflows
Technical Field
The invention relates to the technical field of cloud computing, in particular to an energy-saving scheduling method for improving reliability of multiple workflows.
Background
Cloud computing has received much attention as a new computing model due to its ability to provide distributed computing, storage applications and its on-demand charging. As more and more enterprises and users submit their own application requests to the cloud, the amount of cloud computing scientific workflows that the data center needs to process and the consumed electric energy also increase sharply, and the problem of simultaneous submission of multiple cloud workflows is faced. Generally, different users have different time and reliability requirements, a virtual machine may break down at any time in the actual operation process, and the reliability of the task executed on the virtual machine is difficult to guarantee, so that the reliability of the workflow is reduced, and the reliability requirements of the users are difficult to meet. The traditional mode for improving the workflow scheduling reliability is to adopt a hardware redundancy technology, but the practicability of the system in a data center is greatly reduced due to higher complexity and cost of system construction. Therefore, a more suitable method is to adopt a software technology to design a suitable reliability scheduling scheme, and distribute the tasks to the virtual machines meeting the reliability requirements for execution on the premise of meeting the time constraint of the user. At present, the multi-workflow scheduling method only provides a solution for the situation that a user provides time constraint, cost constraint or both constraints, but no proper solution is provided for the reliability constraint of the user, and the existing multi-workflow reliability scheduling method does not consider the reliability of the multi-workflow.
Therefore, in the traditional scheduling method, the operation energy consumption of the data center is large, the reliability constraint provided by a user is not considered, only the time factor is considered in task sequencing, and the reduction of the workflow reliability caused by machine faults and the like in the actual scheduling process of the tasks is not considered in the task reliability decomposition method, so that the reliability of multiple workflows and the scheduling success rate of application are improved, and meanwhile, the adoption of lower operation energy consumption of the data center is a great challenge for the data center.
Disclosure of Invention
In view of this, the invention provides an energy-saving scheduling method for improving reliability of multiple workflows, which improves reliability of workflows submitted by multiple users and reduces energy consumption of a data center by using a virtual machine frequency modulation technology through two stages of workflow pre-scheduling and task re-scheduling based on reliability and deadline requirements of the users.
The invention relates to an energy-saving scheduling method for improving the reliability of multiple workflows, which comprises the following steps:
step 1, a cloud data center simultaneously receives workflows with deadline constraint DC and reliability constraint RC submitted by a plurality of users, the workflows are sorted in an ascending order according to the size of the DC and stored in a set W, and at the moment, the set of available virtual machines is P;
step 2, selecting the workflow W with the minimum DChPerforming pre-scheduling, comprising the following sub-steps:
step 2.1, calculate workflow W taking into account reliability weight factor and deadline constraintshThe priority weight of each task in the list; w is to behAll tasks in the set T are sorted in descending order according to the priority weight to form a set T;
step 2.2, selecting the task T with the highest priority weight in the task set ThAs the current task to be scheduled and distributing the reliability for the task in real timeA target value;
wherein, the task thTarget value of reliability Rgoal(th) A reliability constraint value RC (t) for the taskh) And the maximum reliability value obtainable for the task across all virtual machines
Figure BDA0002273522030000021
Of a smaller value of p, wherein pkIs the kth available virtual machine; task thIs a reliability constraint value RC (t)h) Comprises the following steps:
Figure BDA0002273522030000022
wherein, AlloT represents a scheduled task set, and UnalloT represents an unscheduled task set; RC (W)h) Represents a workflow WhReliability constraints of (2);
Figure BDA0002273522030000023
product representing the actual reliability of all scheduled tasks, where AR (t)a) Indicating a scheduled task taThe actual reliability of the system; AVR (t)h) Representing a task thAverage reliability across all virtual machines in set P;
Figure BDA0002273522030000031
product representing the average reliability of all unscheduled tasks on all virtual machines in set P, where AVR (t)ua) Indicating an unscheduled task tuaAverage reliability across all virtual machines in set P;
step 2.3, using the earliest completion time in the HEFT and a processor blanking strategy as a task thPre-allocating virtual machines and assigning thDeleting the data in the T, and if the T is not empty, returning to the step 2.2; if T is null, delete W in WhIf W is not empty, returning to the step 2, otherwise obtaining the mapping relation between all tasks and the virtual machine, and executing the step 3;
step 3, judging whether the actual reliability of each workflow in the set W is greater than or equal to the set workflow reliability constraint under the mapping relation between all the tasks and the virtual machines obtained in the step 2, and if so, scheduling by taking the mapping relation between all the tasks and the virtual machines as a final scheduling result; otherwise, performing step 4;
step 4, rescheduling the workflow: readjusting the mapping relation between all tasks and the virtual machine to obtain a final scheduling result for scheduling, and comprising the following substeps:
step 4.1, obtaining the actual start time and the actual completion time of each task under the mapping relation between all the tasks and the virtual machine obtained in the step 2, and storing all the tasks in a set T1 in a descending order according to the actual completion time of the tasks;
step 4.2, selecting the task T with the maximum priority weight value in the set T1currAs the current task to be scheduled; calculating the task t according to the interdependence constraint of the tasks in the directed acyclic graph model, the obtained mapping relation between the tasks and the virtual machine, and the actual start time and the actual finish time of the taskscurrEarliest start time EST (t) on each virtual machinecurr,pk) The latest end time LFT (t)curr,pk) And idle time S on each virtual machinecurr,kFurther calculating to obtain a task tcurrCan be in a virtual machine pkEarliest start time EST' of execution (t)curr,pk) And the latest completion time LFT' (t)curr,pk)；
Step 4.3, for task tcurrAccording to task tcurrObtaining the executable task t at the earliest starting time and the latest finishing time which can be executed by each virtual machine in the P setcurrThe virtual machine set P';
wherein for virtual machine pkIf the difference between the latest end and the earliest start time is greater than or equal to the execution time LFT' (t) of the task on the virtual machinecurr,pk)-EST'(tcurr,pk)≥wcurr,kThen, the current virtual machine p is illustratedkCan perform the task, otherwise, the current virtual machine p is illustratedkThe task cannot be executed, the task t will be executedcurrAdd the virtual machine of (1) to P';
judging whether P' is empty, if not, proceeding step 4.4; if P' is empty, abandon the adjustment of tcurrDirectly compares the current task t in the schedulingcurrThe mapping relation with the virtual machine is used as the final tcurrAnd the mapping relation between the virtual machine and the virtual machine, and executing the step 4.6;
step 4.4, calculate task tcurrIn P', the maximum reliability of each virtual machine is judged whether the maximum reliability is greater than the task tcurrA virtual machine of a reliability target value;
if not, then the task t is directly in the set PcurrSelecting the most reliable virtual machine pmaxAs the target virtual machine, the optimal frequency at this time is the maximum operating frequency f of the virtual machinemax；
If yes, the maximum reliability is greater than the task tcurrThe virtual machine with the reliability target value performs frequency reduction until the task tcurrThe reliability on each virtual machine is equal to the task tcurrThe reliability target value of (2) is equal, and the virtual machine with the minimum energy consumption at the moment is found as the target virtual machine pgoalTarget virtual machine pgoalThe frequency at this time is the optimum frequency fgoal；
Step 4.5, the task tcurrAt target virtual machine pgoalAt the optimum frequency fgoalExecuting the target virtual machine obtained according to the step 4.4 and the optimal frequency f of the virtual machinegoalUpdating the mapping relation between the tasks and the virtual machines in the pre-scheduling; updating the task t based on the mapping relation between the task and the virtual machine in the updated pre-schedulingcurrActual start time AST (t)curr) And actual completion time AFT (t)curr)；
Step 4.6, let tcurrDeleting in the set T1, if T1 is not empty, returning to the step 4.2; and if the T1 is empty, taking the mapping relation between all the current tasks and the virtual machine as a final scheduling result for scheduling.
Wherein, in the step 2, the workflow WhMiddle task tiThe calculation mode of the priority weight is as follows:
step 2.1.1, obtain task tiPredicted start time PST (t)i)：
Figure BDA0002273522030000051
In the formula pred (t)i) For task t in workflow directed acyclic graph modeliSet of parent tasks of, task tjRepresents tiOf the one parent task of (a) is,
Figure BDA0002273522030000052
representing parent tasks tjAverage execution time on a given finite virtual machine, where wj,kRepresents tjIn virtual machine pkM is the size of a given set of virtual machines, TT (t)j,i) Representing a task tjTo tiThe data transmission time of (1);
step 2.1.2, calculate workflow WhThe intermediate predicted start time is less than task tiProduct of average reliability of all tasks predicting start time:
Figure BDA0002273522030000053
wherein, AVR (t)j) Representing a task tjAverage reliability over set P on a given virtual machine;
step 2.1.3, predict task tiRelative processing time on different virtual machines:
Figure BDA0002273522030000054
wherein the content of the first and second substances,
Figure BDA0002273522030000055
respectively representing tasks tiWorkflow W of the sitehGiven a cutoff constraint and a reliability constraint value, R (t)i,pk) Representing a task tiIn virtual machine pkMaximum reliability of operation at maximum frequency, (R (t)i,pk)/(RCWh/PCR(ti) )) represents task tiThe reliability weighting factor of (2);
step 2.1.4, according to predicted task tiObtaining a task t at a relative processing time on a virtual machineiThe relationship between the priority weight and the priority weights of all the descendant tasks of the task is as follows:
Figure BDA0002273522030000056
wherein succ (t)i) Representing a task tiAll children task set of (2), wherein task tsRepresents tiIs a task of one of the children of (1),
Figure BDA0002273522030000057
representing predicted tasks tiThe minimum value of relative processing time at all virtual machines;
step 2.1.5, priority weighting based on export task
Figure BDA0002273522030000061
And task tiThe relationship between the priority weight and the priority weights of all descendant tasks of the task is iterated by using the priority weight of the export task to obtain a task tiThe priority weight of (2).
Wherein the workflow WhThe actual reliability of (2) is calculated as follows:
Figure BDA0002273522030000062
wherein i is 1,2 … N, N is the work flow WhTotal number of tasks in, AR (t)i) As a workflow WhTask t contained iniThe actual reliability of the device.
Wherein, in the step 2.1.2, it is assumed that the failure probability condition of the virtual machine obeys poisson distributionCloth, task tjAverage reliability at a given set of virtual machines P
Figure BDA0002273522030000063
Figure BDA0002273522030000064
Representing tasks in virtual machine pkIn which λkRepresenting a virtual machine pkProbability of failure per unit time, wj,kRepresenting a task tjIn virtual machine pkThe execution time of.
Wherein, use
Figure BDA0002273522030000065
Or
Figure BDA0002273522030000066
Replacement of
Figure BDA0002273522030000067
Where M represents the size of the set of virtual machines.
Has the advantages that:
the invention relates to a multi-workflow energy-saving scheduling algorithm based on two stages of pre-scheduling and re-scheduling, wherein workflow task sequencing and workflow reliability constraint decomposition are optimized in a workflow pre-scheduling stage; and in the task rescheduling stage, according to the task reliability target value obtained in the pre-scheduling stage, a virtual machine frequency modulation technology is used for reselecting the virtual machine which meets the reliability target value and has the minimum energy consumption for the task, and the mapping relation between the optimized task and the virtual machine is obtained, so that the reliability of the multi-workflow is improved, and the operation energy consumption of the data center is reduced.
Drawings
FIG. 1 is a flow chart of workflow pre-scheduling in the energy-saving scheduling method for improving reliability of multiple workflows according to the present invention;
fig. 2 is a flowchart of task rescheduling in the energy-saving scheduling method for improving reliability of multiple workflows according to the present invention.
Detailed Description
The invention is described in detail below by way of example with reference to the accompanying drawings.
The energy-saving scheduling method for improving the reliability of the multiple workflows comprises a workflow pre-scheduling stage and a task re-scheduling stage. FIG. 1 is a flow chart of workflow pre-scheduling in the energy-saving scheduling method for improving reliability of multiple workflows according to the present invention; fig. 2 is a flowchart of task rescheduling in the energy-saving scheduling method for improving reliability of multiple workflows according to the present invention.
In a workflow pre-dispatching stage, predicting the start time of a task, optimizing and calculating a task priority weight by considering cutoff time and reliability weight factors, and giving a mapping relation between the task and a virtual machine by using earliest completion time and a null-insertion algorithm; and calculating and updating the reliability target value of the task to be scheduled in real time according to the actual reliability of the scheduled task to provide a reliability target for a subsequent task rescheduling stage so as to reduce energy consumption as much as possible. The method comprises the steps that by predicting the starting time of tasks, considering the deadline of a workflow and a task reliability weight factor, and adopting an iterative method from bottom to top to calculate, the task sequencing is optimized, and a more reasonable task scheduling sequence is obtained; when the workflow reliability constraint is decomposed, the reliability target value of the current task is updated in real time according to the actual reliability of the scheduled task and the proportional relation between the reliability of the task to be scheduled and the reliability of all tasks not to be scheduled, when the actual reliability of the distributed task caused by the fault of the virtual machine and the like is too low, the reliability target value of the task to be scheduled is calculated and updated in real time, the reliability requirement on the tasks not to be scheduled is improved, and a foundation is provided for reducing the operation energy consumption of a data center as much as possible on the premise that the task reliability constraint is met in a task rescheduling stage.
In the re-scheduling stage, the executable time of the task on the virtual machine is determined according to the workflow deadline constraint and the task dependency relationship, the available virtual machine set is determined according to the reliability target value of the task, the optimal target virtual machine is reallocated for the task and the optimal execution frequency is obtained by using the virtual machine frequency modulation technology, and the mapping relationship between the optimized task and the virtual machine is further updated. And all tasks of the plurality of workflows which are pre-scheduled to be finished are rescheduled. The method comprises the steps of utilizing deadline constraint and virtual machine idle constraint of workflows to obtain the earliest starting time and the latest ending time of tasks on virtual machines, based on a task reliability target value obtained in a pre-scheduling stage, using a virtual machine dynamic frequency modulation technology to reallocate the virtual machines meeting the reliability target value and having the minimum energy consumption for the tasks and obtain the optimal operating frequency of the virtual machines, and further adjusting and updating the mapping relation between the tasks and the virtual machines in multiple workflows, so that the scheduling problem that multiple workflows with reliability constraint are submitted to a cloud data center at the same time and have high operating energy consumption is solved, the scheduling reliability of the multiple workflows is improved, and the energy consumption of the data center is reduced.
In this embodiment, the number of the set multiple workflows is 75, the workflow includes five real workflows, namely linear gebra, Gaussian animation, binary graph, complex tree, and fastfour transform, the number of each workflow is 15, and the number M of available virtual machines is 64. The workflow scheduling algorithm of the embodiment specifically comprises the following steps:
step 1, a cloud data center simultaneously receives workflows with Deadline Constraints (DC) and Reliability Constraints (RC) submitted by a plurality of users, the workflows are sorted in an ascending order according to the size of the DC and stored in a set W, and at the moment, the set of available virtual machines is P;
step 2, selecting the workflow W with the minimum DChPerforming pre-scheduling, comprising the following sub-steps:
step 2.1, calculate workflow W taking into account reliability weight factor and deadline constraintshPriority weight of each task in the workflow WhMiddle task tiThe calculation mode of the priority weight is as follows:
step 2.1.1, obtaining the predicted start time of the task, wherein the task tiPredicted start time PST (t)i) Comprises the following steps:
Figure BDA0002273522030000081
in the formula pred (t)i) For task t in workflow directed acyclic graph modeliSet of parent tasks of, task tjRepresents tiOf the one parent task of (a) is,
Figure BDA0002273522030000082
representing parent tasks tjAverage execution time on a given finite virtual machine, where wj,kRepresents tjIn virtual machine pkM is the size of a given set of virtual machines, TT (t)j,i) Representing a task tjTo tiThe data transmission time of (1). Note the entry task t of the workflowentryIs PST (t)entry) Assume that the initial time of arrival of 75 workflows is 0.
Step 2.1.2, calculate workflow WhThe intermediate predicted start time is less than task tiProduct of average reliability of all tasks predicting start time:
Figure BDA0002273522030000091
in the formula
Figure BDA0002273522030000092
Representing a task tjAt a given average reliability of the set P of virtual machines,
Figure BDA0002273522030000093
representing tasks in virtual machine pkIn which λkRepresenting a virtual machine pkProbability of failure per unit time, wj,kRepresenting a task tjIn virtual machine pkAnd the failure probability condition of the virtual machine is assumed to obey poisson distribution.
Step 2.1.3, according to the deadline of the workflow
Figure BDA0002273522030000094
And reliability constraints
Figure BDA0002273522030000095
Predicting task tiRelative processing time on different virtual machines:
Figure BDA0002273522030000096
in the formula (I), the compound is shown in the specification,
Figure BDA0002273522030000097
respectively representing tasks tiWorkflow W of the sitehGiven a cutoff constraint and a reliability constraint value, R (t)i,pk) Representing a task tiIn virtual machine pkAbove the maximum reliability of operation at the maximum frequency,
Figure BDA0002273522030000098
indicating the predicted idle time remaining for the workflow,
Figure BDA0002273522030000099
indicating the predicted residual reliability of the workflow,
Figure BDA00022735220300000910
representing a task tiThe reliability weighting factor of (2);
step 2.1.4, according to predicted task tiObtaining a task t at a relative processing time on different virtual machinesiThe relationship between the priority weight and the priority weights of all the descendant tasks of the task is as follows:
Figure BDA00022735220300000911
wherein succ (t)i) Representing a task tiAll descendant task sets of, task tsRepresenting a task tiIn the event that one of the child tasks is,
Figure BDA0002273522030000101
for tasks t obtained by predictioniThe minimum value of relative processing time on all virtual machines;
Figure BDA0002273522030000102
the introduction of (1) takes the influence of the cut-off time and the reliability weight factor into consideration, sets higher priority for the task with higher reliability requirement, and optimizes the calculation of the task priority weight, wherein
Figure BDA0002273522030000103
Can also use
Figure BDA0002273522030000104
Or
Figure BDA0002273522030000105
Instead of it.
Step 2.1.5, priority weighting based on export task
Figure BDA0002273522030000106
And task tiThe relationship between the priority weight and the priority weights of all descendant tasks of the task is iterated by using the priority weight of the export task, and the task t is obtained by calculationiThe priority weight of (1); w is to behTask in (2) according to priority weight RankrSorting in descending order and putting into a task set T.
Step 2.2, selecting the task T with the highest priority weight in the task set ThComputing task thIs a reliability constraint value RC (t)h) And a reliability target value Rgoal(th) The following were used:
Figure BDA0002273522030000107
wherein AlloT represents a set of tasks that have been scheduled for completion, UnalloT represents a set of tasks that have not been scheduled,
Figure BDA0002273522030000108
representing the actual reliability product of the scheduled tasks,
Figure BDA0002273522030000109
indicating a task t to be scheduledhAverage reliability across all given virtual machines, where M represents the size of a given set of virtual machines.
Figure BDA00022735220300001010
Indicating an unscheduled task tuaThe average reliability across all of the given virtual machines,
Figure BDA00022735220300001011
representing the average reliability product of the remaining unscheduled tasks,
Figure BDA00022735220300001012
represents the current workflow WhAnd under the reliability constraint, the reliability constraint product of all the unscheduled tasks. According to the real-time scheduling result, the current task t to be scheduled is consideredhIs proportional to the product of the average reliabilities of all unscheduled tasks
Figure BDA00022735220300001013
Updating a task t to be scheduled in real timehThe reliability constraint of the method avoids that the reliability of the whole workflow is too low to meet the user requirement due to the fact that the actual reliability of the distributed tasks is reduced due to the fault of the virtual machine and the like.
According to the actual situation, the task thTarget value of reliability Rgoal(th) Reliability constraint RC (t) obtained for decompositionh) And task thThe smaller value of the maximum reliability achievable on all virtual machines, i.e.
Figure BDA0002273522030000111
The reliability constraint of the tasks to be scheduled is obtained through real-time calculation in the scheduling process, the influence of the reliability change of the allocated tasks on the reliability constraint value of the unallocated tasks is fully considered, when the actual reliability of the scheduled tasks does not meet the requirement, the reliability requirement of the unallocated tasks is improved, the overall reliability requirement of the workflow is met as far as possible, and the defects that the reliability constraint values of all the tasks are directly given out before scheduling, and the task reliability cannot meet the requirement when a machine fault occurs, so that the overall workflow reliability is lower and the user requirement cannot be met in the traditional task reliability decomposition method are overcome.
Step 2.3, using the earliest completion time in the HEFT algorithm and a processor null-insertion strategy to carry out task thPrescheduling, allocating virtual machines and deleting T in ThIf T is not empty, returning to the step 2.2; if T is null, delete W in WhIf W is not empty, returning to the step 2, otherwise obtaining the mapping relation between all tasks and the virtual machine, and executing the step 3 together with the actual start time and the actual finish time of each task;
when the task priority weight is calculated, time and reliability weight factors are considered, longer relative time and higher priority can be distributed to the tasks with higher reliability requirements according to the remaining idle time of the workflow and the reliability weight factors, and the defect that in the traditional task priority ordering, only the influence of time on the task priority ordering is considered, and the task reliability requirements are ignored is avoided.
Step 3, judging whether the actual reliability of each workflow in the set W is greater than or equal to the set workflow reliability constraint under the mapping relation between all the tasks and the virtual machines obtained in the step 2, and if so, scheduling by taking the mapping relation between all the tasks and the virtual machines as a final scheduling result; otherwise, performing step 4; the actual reliability of the workflow obtained by pre-scheduling is shown in formula (6):
Figure BDA0002273522030000112
wherein, AR (W)h)、AR(ti) Are respectively a work flow W in pre-schedulinghReliability and W ofhMiddle task tiThe actual reliability of the device.
Step 4, rescheduling the workflow: readjusting the mapping relation between all tasks and the virtual machine to obtain a final scheduling result for scheduling, and comprising the following substeps:
step 4.1, obtaining the actual start time and the actual completion time of each task under the mapping relation between all the tasks and the virtual machine obtained in the step 2, and storing all the tasks in a set T1 in a descending order according to the actual completion time of the tasks;
step 4.2, selecting the task T with the maximum priority weight value in the set T1currAs the current task to be scheduled; calculating the task t according to the interdependence constraint of the tasks in the directed acyclic graph model, the obtained mapping relation between the tasks and the virtual machine, and the actual start time and the actual finish time of the taskscurrEarliest start time EST (t) on each virtual machinecurr,pk) The latest end time LFT (t)curr,pk) And idle time S on each virtual machinecurr,kFurther calculating to obtain a task tcurrCan be in a virtual machine pkEarliest start time EST' of execution (t)curr,pk)＝max{EST(tcurr,pk),ts(Scurr,k) } and the latest completion time LFT' (t)curr,pk)＝min{LFT(tcurr,pk),te(Scurr,k)}；
Step 4.3, for task tcurrAccording to task tcurrObtaining the executable task t at the earliest starting time and the latest finishing time of the execution of each virtual machine in the P setcurrThe virtual machine set P';
wherein for virtual machine pkIf the difference between the latest end and the earliest start time is greater than or equal to the execution time LFT' (t) of the task on the virtual machinecurr,pk)-EST'(tcurr,pk)≥wcurr,kThen, the current virtual machine p is illustratedkCan perform the task, otherwise, the current virtual machine p is illustratedkThe task cannot be executed, the task t will be executedcurrAdd the virtual machine of (1) to P';
judgment of pkIf the value is not null, performing the step 4.4; if P' is empty, abandon the adjustment of tcurrDirectly obtain the task t in the current schedulingcurrThe mapping relation with the virtual machine is used as the final tcurrAnd the mapping relation between the virtual machine and the virtual machine, and executing the step 4.6;
step 4.4, calculate task tcurrIn P', the maximum reliability of each virtual machine is judged whether the maximum reliability is greater than the task tcurrReliability target value Rgoal(tcurr) The virtual machine of (1);
if not, then the task t is directly in the set PcurrSelecting the most reliable virtual machine pmaxAs a target virtual machine pgoalOptimum frequency f at this timegoalFor the maximum operating frequency f of the virtual machinemax；
If yes, the maximum reliability is greater than the task tcurrThe virtual machine with the reliability target value performs frequency reduction until the task tcurrThe reliability on each virtual machine is equal to the task tcurrThe reliability target value of (2) is equal, and the virtual machine with the minimum energy consumption at the moment is found as the target virtual machine pgoalTarget virtual machine pgoalThe frequency at this time is the optimum frequency fgoal(ii) a This step can be performed while the task t is satisfiedcurrOn the basis of the reliability target value, the spare time of the cut-off time is fully utilized, and the energy consumption of the data center is reduced to the maximum extent through the technology of dynamically adjusting the frequency of the virtual machine.
Step 4.5, the task tcurrAt target virtual machine pgoalAt the optimum frequency fgoalExecuting the target virtual machine obtained according to the step 4.4 and the optimal frequency f of the virtual machinegoalUpdating the mapping relation between the tasks and the virtual machines in the pre-scheduling; updating the task t based on the mapping relation between the task and the virtual machine in the updated pre-schedulingcurrActual start time AST (t)curr) And actual completion time AFT (t)curr)；
Step 4.6, let tcurrDeleting in the set T1, if T1 is not empty, returning to the step 4.2; and if the T1 is empty, taking the mapping relation between all the current tasks and the virtual machine as a final scheduling value for scheduling.
By the method, the approximately optimal mapping relation between the tasks and the virtual machines can be obtained under the condition that multiple workflows have reliability constraint, the energy consumption of cloud data center operation is reduced, and the reliability of multiple workflow scheduling and the service success rate of the cloud data center are improved.
In summary, the above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (5)

1. An energy-saving scheduling method for improving reliability of multiple workflows is characterized by comprising the following steps:
step 1, a cloud data center simultaneously receives workflows with deadline constraint DC and reliability constraint RC submitted by a plurality of users, the workflows are sorted in an ascending order according to the size of the DC and stored in a set W, and at the moment, the set of available virtual machines is P;
step 2, selecting the workflow W with the minimum DChPerforming pre-scheduling, comprising the following sub-steps:
step 2.1, calculate workflow W taking into account reliability weight factor and deadline constraintshThe priority weight of each task in the list; w is to behAll tasks in the set T are sorted in descending order according to the priority weight to form a set T;
step 2.2, selecting the task T with the highest priority weight in the task set ThAs the current task to be scheduled, and distributing a reliability target value for the task in real time;
wherein, the task thTarget value of reliability Rgoal(th) A reliability constraint value RC (t) for the taskh) And the maximum reliability value obtainable for the task across all virtual machines
Figure FDA0002273522020000011
Of a smaller value of p, wherein pkIs the kth available virtual machine; task thIs a reliability constraint value RC (t)h) Comprises the following steps:
Figure FDA0002273522020000012
wherein, AlloT represents a scheduled task set, and UnalloT represents an unscheduled task set; RC (W)h) Represents a workflow WhReliability constraints of (2);
Figure FDA0002273522020000013
product representing the actual reliability of all scheduled tasks, where AR (t)a) Indicating a scheduled task taThe actual reliability of the system; AVR (t)h) Representing a task thAverage reliability across all virtual machines in set P;
Figure FDA0002273522020000014
product representing the average reliability of all unscheduled tasks on all virtual machines in set P, where AVR (t)ua) Indicating an unscheduled task tuaAverage reliability across all virtual machines in set P;
step 2.3, using the earliest completion time in the HEFT and a processor blanking strategy as a task thPre-allocating virtual machines and assigning thDeleting the data in the T, and if the T is not empty, returning to the step 2.2; if T is null, delete W in WhIf W is not empty, returning to the step 2, otherwise obtaining the mapping relation between all tasks and the virtual machine, and executing the step 3;
step 3, judging whether the actual reliability of each workflow in the set W is greater than or equal to the set workflow reliability constraint under the mapping relation between all the tasks and the virtual machines obtained in the step 2, and if so, scheduling by taking the mapping relation between all the tasks and the virtual machines as a final scheduling result; otherwise, performing step 4;
step 4, rescheduling the workflow: readjusting the mapping relation between all tasks and the virtual machine to obtain a final scheduling result for scheduling, and comprising the following substeps:
step 4.1, obtaining the actual start time and the actual completion time of each task under the mapping relation between all the tasks and the virtual machine obtained in the step 2, and storing all the tasks in a set T1 in a descending order according to the actual completion time of the tasks;
step 4.2, selecting the task T with the maximum priority weight value in the set T1currAs the current task to be scheduled; calculating the task t according to the interdependence constraint of the tasks in the directed acyclic graph model, the obtained mapping relation between the tasks and the virtual machine, and the actual start time and the actual finish time of the taskscurrEarliest start time EST (t) on each virtual machinecurr,pk) The latest end time LFT (t)curr,pk) And idle time S on each virtual machinecurr,kFurther calculating to obtain a task tcurrCan be in a virtual machine pkEarliest start time EST' of execution (t)curr,pk) And the latest completion time LFT' (t)curr,pk)；
Step 4.3, for task tcurrAccording to task tcurrObtaining the executable task t at the earliest starting time and the latest finishing time which can be executed by each virtual machine in the P setcurrThe virtual machine set P';
wherein for virtual machine pkIf the difference between the latest end and the earliest start time is greater than or equal to the execution time LFT' (t) of the task on the virtual machinecurr,pk)-EST'(tcurr,pk)≥wcurr,kThen, the current virtual machine p is illustratedkCan perform the task, otherwise, the current virtual machine p is illustratedkThe task cannot be executed, the task t will be executedcurrAdd the virtual machine of (1) to P';
judging whether P' is empty, if not, proceeding step 4.4; if P' is empty, abandon the adjustment of tcurrDirectly compares the current task t in the schedulingcurrThe mapping relation with the virtual machine is used as the final tcurrAnd the mapping relation between the virtual machine and the virtual machine, and executing the step 4.6;
step 4.4, calculate task tcurrIn P', the maximum reliability of each virtual machine is judged whether the maximum reliability is greater than the task tcurrA virtual machine of a reliability target value;
if not, then the task t is directly in the set PcurrSelecting the most reliable virtual machine pmaxAs the target virtual machine, the optimal frequency at this time is the maximum operating frequency f of the virtual machinemax；
If yes, the maximum reliability is greater than the task tcurrThe virtual machine with the reliability target value performs frequency reduction until the task tcurrThe reliability on each virtual machine is equal to the task tcurrThe reliability target value of (2) is equal, and the virtual machine with the minimum energy consumption at the moment is found as the target virtual machine pgoalTarget virtual machine pgoalThe frequency at this time is the optimum frequency fgoal；
Step 4.5, the task tcurrAt target virtual machine pgoalAt the optimum frequency fgoalExecuting the target virtual machine obtained according to the step 4.4 and the optimal frequency f of the virtual machinegoalUpdating the mapping relation between the tasks and the virtual machines in the pre-scheduling; updating the task t based on the mapping relation between the task and the virtual machine in the updated pre-schedulingcurrActual start time AST (t)curr) And actual completion time AFT (t)curr)；
Step 4.6, let tcurrDeleting in the set T1, if T1 is not empty, returning to the step 4.2; and if the T1 is empty, scheduling the mapping relation between all the tasks and the virtual machine as a final scheduling result.
2. The energy-saving scheduling method for improving reliability of multiple workflows as claimed in claim 1, wherein in the step 2, the workflow WhMiddle task tiThe calculation mode of the priority weight is as follows:
step 2.1.1, obtain task tiPredicted start time PST (t)i)：
Figure FDA0002273522020000031
In the formula (a), (b) pred(ti) For task t in workflow directed acyclic graph modeliSet of parent tasks of, task tjRepresents tiOf the one parent task of (a) is,
Figure FDA0002273522020000041
representing parent tasks tjAverage execution time on a given finite virtual machine, where wj,kRepresents tjIn virtual machine pkM is the size of a given set of virtual machines, TT (t)j,i) Representing a task tjTo tiThe data transmission time of (1);
step 2.1.2, calculate workflow WhThe intermediate predicted start time is less than task tiProduct of average reliability of all tasks predicting start time:
Figure FDA0002273522020000042
wherein, AVR (t)j) Representing a task tjAverage reliability over set P on a given virtual machine;
step 2.1.3, predict task tiRelative processing time on different virtual machines:
Figure FDA0002273522020000043
wherein the content of the first and second substances,
Figure FDA0002273522020000044
respectively representing tasks tiWorkflow W of the sitehGiven a cutoff constraint and a reliability constraint value, R (t)i,pk) Representing a task tiIn virtual machine pkAbove the maximum reliability of operation at the maximum frequency,
Figure FDA0002273522020000045
representing a task tiThe reliability weighting factor of (2);
step 2.1.4, according to predicted task tiObtaining a task t at a relative processing time on a virtual machineiThe relationship between the priority weight and the priority weights of all the descendant tasks of the task is as follows:
Figure FDA0002273522020000046
wherein succ (t)i) Representing a task tiAll children task set of (2), wherein task tsRepresents tiIs a task of one of the children of (1),
Figure FDA0002273522020000047
representing predicted tasks tiThe minimum value of relative processing time at all virtual machines;
step 2.1.5, priority weighting based on export task
Figure FDA0002273522020000048
And task tiThe relationship between the priority weight and the priority weights of all descendant tasks of the task is iterated by using the priority weight of the export task to obtain a task tiThe priority weight of (2).
3. The power-save scheduling method for improving reliability of multiple workflows according to claim 1,
the workflow WhThe actual reliability of (2) is calculated as follows:
Figure FDA0002273522020000051
wherein i is 1,2 … N, N is the work flow WhTotal number of tasks in, AR (t)i) As a workflow WhTask t contained iniThe actual reliability of the device.
4. The power-saving scheduling method for improving reliability of multiple workflows as claimed in claim 1, which comprisesCharacterized in that in the step 2.1.2, the fault probability condition of the virtual machine is assumed to obey Poisson distribution, and the task tjAverage reliability at a given set of virtual machines P
Figure FDA0002273522020000052
Figure FDA0002273522020000053
Representing tasks in virtual machine pkIn which λkRepresenting a virtual machine pkProbability of failure per unit time, wj,kRepresenting a task tjIn virtual machine pkThe execution time of.
5. The energy-efficient scheduling method for improving reliability of multiple workflows according to claim 1 wherein
Figure FDA0002273522020000054
Or
Figure FDA0002273522020000055
Replacement of
Figure FDA0002273522020000056
Where M represents the size of the set of virtual machines.
CN201911113879.XA 2019-11-14 2019-11-14 Energy-saving scheduling method for improving reliability of multiple workflows Active CN110908772B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911113879.XA CN110908772B (en) 2019-11-14 2019-11-14 Energy-saving scheduling method for improving reliability of multiple workflows

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911113879.XA CN110908772B (en) 2019-11-14 2019-11-14 Energy-saving scheduling method for improving reliability of multiple workflows

Publications (2)

Publication Number Publication Date
CN110908772A true CN110908772A (en) 2020-03-24
CN110908772B CN110908772B (en) 2022-09-06

Family

ID=69817357

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911113879.XA Active CN110908772B (en) 2019-11-14 2019-11-14 Energy-saving scheduling method for improving reliability of multiple workflows

Country Status (1)

Country Link
CN (1) CN110908772B (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111650878A (en) * 2020-06-15 2020-09-11 北京理工大学 Method for optimizing programmability of flow when multiple controllers in software defined network fail
CN113064711A (en) * 2021-04-15 2021-07-02 北京理工大学 Online multi-workflow dynamic scheduling method and system
CN113537721A (en) * 2021-06-21 2021-10-22 华南师范大学 Control method, system, and medium for business workflow local time constraint adjustment
CN113722076A (en) * 2021-10-29 2021-11-30 华南理工大学 Real-time workflow scheduling method based on QoS and energy consumption collaborative optimization

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120096466A1 (en) * 2009-02-05 2012-04-19 Wei Sun Method, system and program for deadline constrained task admission control and scheduling using genetic approach
US8583467B1 (en) * 2012-08-23 2013-11-12 Fmr Llc Method and system for optimized scheduling of workflows
CN105912383A (en) * 2016-05-05 2016-08-31 中国人民解放军国防科学技术大学 High-reliability dependent task scheduling and resource configuration method
CN107908464A (en) * 2017-10-12 2018-04-13 华东师范大学 A kind of cloud computing workflow energy-saving scheduling method for considering reliability
CN109948848A (en) * 2019-03-19 2019-06-28 中国石油大学(华东) Research-on-research flows down the Cost Optimization dispatching method of deadline constraint in a kind of cloud
CN110377411A (en) * 2019-07-22 2019-10-25 郑州轻工业学院 A kind of the workflow task dispatching method and system of Based on Distributed cloud

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120096466A1 (en) * 2009-02-05 2012-04-19 Wei Sun Method, system and program for deadline constrained task admission control and scheduling using genetic approach
US8583467B1 (en) * 2012-08-23 2013-11-12 Fmr Llc Method and system for optimized scheduling of workflows
CN105912383A (en) * 2016-05-05 2016-08-31 中国人民解放军国防科学技术大学 High-reliability dependent task scheduling and resource configuration method
CN107908464A (en) * 2017-10-12 2018-04-13 华东师范大学 A kind of cloud computing workflow energy-saving scheduling method for considering reliability
CN109948848A (en) * 2019-03-19 2019-06-28 中国石油大学(华东) Research-on-research flows down the Cost Optimization dispatching method of deadline constraint in a kind of cloud
CN110377411A (en) * 2019-07-22 2019-10-25 郑州轻工业学院 A kind of the workflow task dispatching method and system of Based on Distributed cloud

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
景维鹏等: "多ＤＡＧ工作流在云计算环境下的可靠性调度方法", 《西安电子科技大学学报（自然科学版）》 *
郑宏升: "基于萤火虫算法和动态优先级的多ＱｏＳ", 《计算机集成制造系统》 *

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111650878A (en) * 2020-06-15 2020-09-11 北京理工大学 Method for optimizing programmability of flow when multiple controllers in software defined network fail
CN111650878B (en) * 2020-06-15 2021-05-04 北京理工大学 Method for optimizing programmability of flow when multiple controllers in software defined network fail
CN113064711A (en) * 2021-04-15 2021-07-02 北京理工大学 Online multi-workflow dynamic scheduling method and system
CN113064711B (en) * 2021-04-15 2022-09-20 北京理工大学 Online multi-workflow dynamic scheduling method and system
CN113537721A (en) * 2021-06-21 2021-10-22 华南师范大学 Control method, system, and medium for business workflow local time constraint adjustment
CN113722076A (en) * 2021-10-29 2021-11-30 华南理工大学 Real-time workflow scheduling method based on QoS and energy consumption collaborative optimization
CN113722076B (en) * 2021-10-29 2022-03-29 华南理工大学 Real-time workflow scheduling method based on QoS and energy consumption collaborative optimization

Also Published As

Publication number Publication date
CN110908772B (en) 2022-09-06

Similar Documents

Publication Publication Date Title
CN110908772B (en) Energy-saving scheduling method for improving reliability of multiple workflows
CN108009023B (en) Task scheduling method based on BP neural network time prediction in hybrid cloud
CN107329815B (en) BP-Tabu search-based cloud task load balancing scheduling method
CN110297699B (en) Scheduling method, scheduler, storage medium and system
CN109582448B (en) Criticality and timeliness oriented edge calculation task scheduling method
CN101237469B (en) Method for optimizing multi-QoS grid workflow based on ant group algorithm
CN104765640B (en) A kind of intelligent Service dispatching method
CN108154317B (en) Workflow group scheduling method based on example self-adaptive distribution integration in multi-cloud environment
CN109255481A (en) A kind of distribution inventory dispatches system and improved method
CN104023042B (en) Cloud platform resource scheduling method
CN110262897B (en) Hadoop calculation task initial allocation method based on load prediction
CN111722910A (en) Cloud job scheduling and resource allocation method
CN110308967A (en) A kind of workflow cost based on mixed cloud-delay optimization method for allocating tasks
CN109710392A (en) A kind of heterogeneous resource dispatching method based on mixed cloud
CN108628672A (en) Method for scheduling task, system, terminal device and storage medium
CN107885577A (en) A kind of task based access control primary copy and the hybrid fault-tolerant scheduling method of cloud computing resources for bringing up again knot conjunction
CN111782627B (en) Task and data cooperative scheduling method for wide-area high-performance computing environment
CN109032756A (en) Scheduling method of virtualized cloud data center
CN113689122A (en) Multi-target combination scheduling method considering transportation equipment in cloud-edge collaborative environment
CN106934539A (en) It is a kind of with limited and expense restriction workflow schedule method
CN110084507B (en) Scientific workflow scheduling optimization method based on hierarchical perception in cloud computing environment
CN108170523A (en) A kind of Random Task sequence dispatching method of mobile cloud computing
CN112306642B (en) Workflow scheduling method based on stable matching game theory
CN110751292B (en) Cloud manufacturing multi-view collaborative scheduling optimization method based on genetic algorithm
CN105094989B (en) Adaptive QoS granularity division method based on fuzzy clustering

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant