CN113127205B - Workflow scheduling method meeting deadline constraint and optimizing cost in cloud - Google Patents

Workflow scheduling method meeting deadline constraint and optimizing cost in cloud Download PDF

Info

Publication number
CN113127205B
CN113127205B CN202110477582.2A CN202110477582A CN113127205B CN 113127205 B CN113127205 B CN 113127205B CN 202110477582 A CN202110477582 A CN 202110477582A CN 113127205 B CN113127205 B CN 113127205B
Authority
CN
China
Prior art keywords
task
workflow
virtual machine
deadline
sub
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110477582.2A
Other languages
Chinese (zh)
Other versions
CN113127205A (en
Inventor
卢政昊
潘纪奎
曹建建
王子健
孙福权
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Northeastern University Qinhuangdao Branch
Original Assignee
Northeastern University Qinhuangdao Branch
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Northeastern University Qinhuangdao Branch filed Critical Northeastern University Qinhuangdao Branch
Priority to CN202110477582.2A priority Critical patent/CN113127205B/en
Publication of CN113127205A publication Critical patent/CN113127205A/en
Application granted granted Critical
Publication of CN113127205B publication Critical patent/CN113127205B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G06F9/5077—Logical partitioning of resources; Management or configuration of virtualized resources
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/45595—Network integration; Enabling network access in virtual machine instances
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/50—Indexing scheme relating to G06F9/50
    • G06F2209/5011—Pool
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The invention discloses a workflow scheduling method meeting deadline constraints and optimizing cost in cloud, which comprises the following steps: step 1: assigning deadlines for the entire workflow of the servers in the cloud to each task based on the delta-alap, thereby forming sub-deadlines; step 2: sequencing each task in the workflow based on the sub-deadline so as to form an ordered task queue; and step 3: and allocating a virtual machine to each task in the task queue in turn, so that the virtual machine meets the deadline constraint and the cost is reduced. The workflow scheduling method effectively controls the completion time of the workflow and optimizes the cost.

Description

Workflow scheduling method meeting deadline constraint and optimizing cost in cloud
Technical Field
The invention belongs to the technical field of workflow scheduling and cost optimization in a cloud server, and relates to a workflow scheduling method which meets deadline constraints and optimizes cost in a cloud.
Background
Workflows are often used for large-scale modeling scientific issues such as bioinformatics, astronomy, and physics. Such workflows require ever-increasing data and computing, and thus a high-performance computing environment is needed to execute the workflow in a reasonable amount of time. For many years, extensive research has been conducted on workflow scheduling in environments such as grids and clusters. However, with the advent of cloud computing, new approaches need to be developed to deal with workflow scheduling in cloud servers.
Workflow scheduling is now a widely studied subject of cloud computing, as optimizing workflow scheduling can greatly improve the overall performance of cloud computing. The key to optimizing a workflow is the scheduling of tasks in the workflow, which is an NP-hard problem. In the cloud, service providers offer resources of different performance at different prices. Insufficient resource allocation will inevitably impair service performance, while excessive resource allocation may lead to unnecessary costs. Also, generally speaking, better performing resources will run faster and have a shorter completion time than less performing resources, but will also be more expensive. Thus, the same workflow, with different resources deployed, may result in different completion times and costs. Then, for a specific workflow application, how to optimize the execution cost on the premise of meeting the deadline constraint given by the user is an important problem facing us.
Disclosure of Invention
In order to solve the technical problems, the invention aims to provide a workflow scheduling method which meets the deadline constraint and optimizes the cost in the cloud.
The invention provides a workflow scheduling method which meets deadline constraints and optimizes cost in cloud, comprising the following steps:
step 1: assigning deadlines for the entire workflow of the servers in the cloud to each task based on the delta-alap, thereby forming sub-deadlines;
step 2: sequencing each task in the workflow based on the sub-deadline so as to form an ordered task queue;
and 3, step 3: and allocating a virtual machine to each task in the task queue in turn, so that the virtual machine meets the deadline constraint and the cost is reduced.
In the workflow scheduling method satisfying the deadline constraint and optimizing the cost in the cloud of the present invention, the step 1 specifically includes:
step 1-1: computing task t according toiThe alap value of (c):
Figure BDA0003047851650000021
wherein, alapiIs tiThe alap value represents a measure of how long the execution start time of the task can be delayed on the premise of not affecting the critical path of the workflow, and the initial value of the alap is determined according to the task executed first by the workflow; t is tjIs tiA subtask of (1), task tjMust be at task tiThe execution can be started after the completion of the data transmissioni,jRepresentative task tiSent to task tjThe size of the data of (a); bw is the bandwidth between tasks, wiIs tiThe computational effort of s*For the fastest executing virtual machine, p(s)*) Is s is*The execution speed of (2);
step 1-2: computing task tiSub-cut-off time sd ofi：
Figure BDA0003047851650000022
Wherein D is the deadline appointed by the user, and CPL is the critical path length of the workflow;
step 1-3: computing task tjDelta ofjThe value:
Figure BDA0003047851650000023
wherein, deltajIs a representation calculation delta-alapjWhether or not t is consideredjIs a function of a random number with a return value of [0,1 ], datai,jThe larger is/bw, δjThe greater the probability of returning 0;
step 1-4: the delta-alap value for the task is calculated according to the following equation:
Figure BDA0003047851650000031
step 1-5: calculate the sub-deadlines for each task:
Figure BDA0003047851650000032
wherein, δ -sdiIndicating the sub-deadlines for the ith task.
In the workflow scheduling method satisfying the deadline constraint and optimizing the cost in the cloud of the present invention, the step 3 specifically includes:
step 3-1: the candidate virtual machines include all virtual machines that have been used in the solution, and virtual machines that have not been used but can be added to the sequence at any time, the first criterion of virtual machine selection is to select a virtual machine that meets the sub-deadline and minimizes the cost increase;
step 3-2: when no virtual machine can meet the sub-deadline, the criterion for selecting the virtual machine is to select the virtual machine with the shortest time for completing the task from the resource pool;
step 3-3: if the selected virtual machine is not the fastest type, an attempt is made to set its type to a faster level and update the completion time of each task deployed thereon.
The invention aims to provide a workflow scheduling method which meets deadline constraints and optimizes cost in cloud. Through the three-stage task scheduling process, the completion time of the workflow is effectively controlled, and the cost is optimized. Experimental results show that DCCO has the highest success rate, meets the constraint of the deadline, and can optimize the execution cost.
Drawings
FIG. 1 is a flow chart of a method for workflow scheduling in a cloud that satisfies a deadline constraint and optimizes costs in accordance with the present invention.
Detailed Description
The invention aims to find a cheaper cloud workflow scheduling method on the premise of meeting the deadline constraint given by a user. A workflow scheduling method which meets deadline constraints and optimizes cost in a cloud environment is provided.
The cloud computing uses virtualization technology to virtualize various resources in the cloud platform into resource pools for unified management and service. The IaaS provider provides the user with a virtual machine S ═ { S } similar to Amazon EC21,s2,…,sl…, these virtual machines have different processing speeds and costs. Generally, the faster the processing speed of a virtual machine, the higher the cost. Conversely, the slower the processing speed of the virtual machine, the lower the cost.
The invention provides a workflow scheduling method which meets deadline constraints and optimizes cost in a cloud environment. The strategy consists of three stages of distribution deadline, task sequencing and virtual machine selection. Through the three-stage task scheduling process, the completion time of the workflow is effectively controlled, and the cost is optimized. Experimental results show that the method has the highest success rate, meets the constraint of the deadline, and can optimize the execution cost. The method specifically comprises the following steps:
step 1: assigning deadlines for the entire workflow of the servers in the cloud to each task based on the delta-alap, thereby forming sub-deadlines;
the step 1 specifically comprises:
step 1-1: computing task t according toiThe alap value of (c):
Figure BDA0003047851650000041
wherein, alapiIs tiThe alap value represents a measure of how long the execution start time of the task can be delayed on the premise of not affecting the critical path of the workflow, and the initial value of the alap is determined according to the task executed first by the workflow; t is tjIs tiA subtask of (1), task tjMust be at task tiThe execution can be started after the completion of the data transmissioni,jRepresentative task tiSent to task tjThe size of the data of (a); bw is the bandwidth between tasks, wiIs tiThe computational effort of s*For the fastest executing virtual machine, p(s)*) Is s is*The execution speed of (2);
step 1-2: computing task tiSub-cut-off time sd ofi：
Figure BDA0003047851650000042
Wherein D is the deadline appointed by the user, and CPL is the critical path length of the workflow;
step 1-3: computing task tjDelta ofjThe value:
Figure BDA0003047851650000051
wherein, deltajIs a representation calculation delta-alapjWhether or not t is consideredjIs a function of a random number with a return value of [0,1 ], datai,jThe larger is/bw, δjThe greater the probability of returning 0;
step 1-4: the delta-alap value for the task is calculated according to the following equation:
Figure BDA0003047851650000052
step 1-5: calculate the sub-deadlines for each task:
Figure BDA0003047851650000053
wherein, δ -sdiIndicating the sub-deadlines for the ith task.
Step 2: sequencing each task in the workflow based on the sub-deadline so as to form an ordered task queue;
and 3, step 3: allocating a virtual machine to each task in the task queue in sequence to enable the virtual machine to meet deadline constraints and reduce cost, wherein the step 3 specifically comprises:
step 3-1: the candidate virtual machines include all virtual machines that have been used in the solution, and virtual machines that have not been used but can be added to the sequence at any time, the first criterion of virtual machine selection is to select a virtual machine that meets the sub-deadline and minimizes the cost increase;
step 3-2: when no virtual machine can meet the sub-deadline, the criterion for selecting the virtual machine is to select the virtual machine with the shortest time for completing the task from the resource pool;
step 3-3: if the selected virtual machine is not the fastest type, an attempt is made to set its type to a faster level and update the completion time of each task deployed thereon.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the scope of the present invention, which is defined by the appended claims.

Claims (2)

1. A workflow scheduling method in a cloud that satisfies a deadline constraint and optimizes costs, comprising:
step 1: assigning deadlines for the entire workflow of the servers in the cloud to each task based on the delta-alap, thereby forming sub-deadlines;
step 2: sequencing each task in the workflow based on the sub-deadline so as to form an ordered task queue;
and step 3: allocating a virtual machine to each task in the task queue in sequence to enable the virtual machine to meet the deadline constraint and reduce the cost;
the step 1 specifically comprises:
step 1-1: computing task t according toiThe alap value of (c):
Figure FDA0003498339550000011
wherein, alapiIs tiThe alap value represents a measure of how long the execution start time of the task can be delayed on the premise of not affecting the critical path of the workflow, and the initial value of the alap is determined according to the task executed first by the workflow; t is tjIs tiA subtask of (1), task tjMust be at task tiThe execution can be started after the completion of the data transmissioni,jRepresentative task tiSent to task tjThe size of the data of (a); bw is that of the taskBandwidth of (w)iIs tiThe computational effort of s*For the fastest executing virtual machine, p(s)*) Is s is*The execution speed of (2);
step 1-2: computing task tiSub-cut-off time sd ofi：
Figure FDA0003498339550000012
Wherein D is the deadline appointed by the user, and CPL is the critical path length of the workflow;
step 1-3: computing task tjDelta ofjThe value:
Figure FDA0003498339550000013
wherein, deltajIs a representation calculation delta-alapjWhether or not t is consideredjIs a function of a random number with a return value of [0,1 ], datai,jThe larger is/bw, δjThe greater the probability of returning 0;
step 1-4: the delta-alap value for the task is calculated according to the following equation:
Figure FDA0003498339550000021
step 1-5: calculate the sub-deadlines for each task:
Figure FDA0003498339550000022
wherein, δ -sdiIndicating the sub-deadlines for the ith task.
2. The method for workflow scheduling in cloud that satisfies deadline constraints and optimizes costs according to claim 1, wherein said step 3 specifically comprises:
step 3-1: the candidate virtual machines include all virtual machines that have been used in the solution, and virtual machines that have not been used but can be added to the sequence at any time, the first criterion of virtual machine selection is to select a virtual machine that meets the sub-deadline and minimizes the cost increase;
step 3-2: when no virtual machine can meet the sub-deadline, the criterion for selecting the virtual machine is to select the virtual machine with the shortest time for completing the task from the resource pool;
step 3-3: if the selected virtual machine is not the fastest type, an attempt is made to set its type to a faster level and update the completion time of each task deployed thereon.
CN202110477582.2A 2021-04-30 2021-04-30 Workflow scheduling method meeting deadline constraint and optimizing cost in cloud Active CN113127205B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110477582.2A CN113127205B (en) 2021-04-30 2021-04-30 Workflow scheduling method meeting deadline constraint and optimizing cost in cloud

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110477582.2A CN113127205B (en) 2021-04-30 2021-04-30 Workflow scheduling method meeting deadline constraint and optimizing cost in cloud

Publications (2)

Publication Number Publication Date
CN113127205A CN113127205A (en) 2021-07-16
CN113127205B true CN113127205B (en) 2022-05-17

Family

ID=76780619

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110477582.2A Active CN113127205B (en) 2021-04-30 2021-04-30 Workflow scheduling method meeting deadline constraint and optimizing cost in cloud

Country Status (1)

Country Link
CN (1) CN113127205B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103473122A (en) * 2013-08-21 2013-12-25 上海交通大学 Workflow system resource scheduling method in cloud computing environment
CN107924345A (en) * 2015-06-26 2018-04-17 亚马逊技术股份有限公司 Data storage area for the polymerization measurement result of measurement
CN109948848A (en) * 2019-03-19 2019-06-28 中国石油大学(华东) Research-on-research flows down the Cost Optimization dispatching method of deadline constraint in a kind of cloud
CN110609736A (en) * 2019-07-30 2019-12-24 中国人民解放军国防科技大学 Deadline constraint scientific workflow scheduling method in cloud environment
CN112256402A (en) * 2020-10-30 2021-01-22 深圳供电局有限公司 Cloud platform data center resource prediction and scheduling method and system

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10635845B2 (en) * 2018-04-16 2020-04-28 Mellanox Technologies, Ltd. Method and apparatus for improving Boolean satisfiability solver generated based on input design with data qualifier signals
CN109784663B (en) * 2018-12-20 2022-11-25 西北大学 Workflow scheduling method and device
CN112149178B (en) * 2020-09-18 2022-07-15 江苏方天电力技术有限公司 Privacy protection workflow issuing method for maintaining availability of critical path

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103473122A (en) * 2013-08-21 2013-12-25 上海交通大学 Workflow system resource scheduling method in cloud computing environment
CN107924345A (en) * 2015-06-26 2018-04-17 亚马逊技术股份有限公司 Data storage area for the polymerization measurement result of measurement
CN109948848A (en) * 2019-03-19 2019-06-28 中国石油大学(华东) Research-on-research flows down the Cost Optimization dispatching method of deadline constraint in a kind of cloud
CN110609736A (en) * 2019-07-30 2019-12-24 中国人民解放军国防科技大学 Deadline constraint scientific workflow scheduling method in cloud environment
CN112256402A (en) * 2020-10-30 2021-01-22 深圳供电局有限公司 Cloud platform data center resource prediction and scheduling method and system

Also Published As

Publication number Publication date
CN113127205A (en) 2021-07-16

Similar Documents

Publication Publication Date Title
US10223165B2 (en) Scheduling homogeneous and heterogeneous workloads with runtime elasticity in a parallel processing environment
JP5939740B2 (en) Method, system and program for dynamically allocating resources
CN109710372B (en) Calculation intensive cloud workflow scheduling method based on owl search algorithm
Zhou et al. Concurrent workflow budget-and deadline-constrained scheduling in heterogeneous distributed environments
Singh et al. A comparative study of various scheduling algorithms in cloud computing
Stavrinides et al. Security and cost aware scheduling of real-time IoT workflows in a mist computing environment
CN114675953A (en) Resource dynamic scheduling method, device, equipment and computer readable storage medium
Haladu et al. Optimizing task scheduling and resource allocation in cloud data center, using enhanced min-min algorithm
Wu et al. Abp scheduler: Speeding up service spread in docker swarm
CN113127205B (en) Workflow scheduling method meeting deadline constraint and optimizing cost in cloud
Komarasamy et al. ScHeduling of jobs and Adaptive Resource Provisioning (SHARP) approach in cloud computing
CN115858169A (en) Operation resource allocation method and device, electronic equipment and storage medium
Wang et al. Cooperative job scheduling and data allocation in data-intensive parallel computing clusters
CN112698911B (en) Cloud job scheduling method based on deep reinforcement learning
Mala et al. Resource allocation in cloud using enhanced max-min algorithm
Alatawi et al. Hybrid load balancing approach based on the integration of QoS and power consumption in cloud computing
Haque et al. A priority-based process scheduling algorithm in cloud computing
CN112306642A (en) Workflow scheduling method based on stable matching game theory
Postoaca et al. h-Fair: asymptotic scheduling of heavy workloads in heterogeneous data centers
CN112308304B (en) Workflow execution time optimization method and device
CN116893893B (en) Virtual machine scheduling method and device, electronic equipment and storage medium
Patil et al. Review on a comparative study of various task scheduling algorithm in cloud computing environment
EP4235424A1 (en) Resource control method for function computing, device, and medium
Weihua et al. Analysis of information management and scheduling technology in Hadoop
Liu et al. Online deadline-constrained scheduling of parallel applications in CPSs

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant