CN110704180B - Workflow scheduling method based on hybrid cloud - Google Patents
Workflow scheduling method based on hybrid cloud Download PDFInfo
- Publication number
- CN110704180B CN110704180B CN201910852597.5A CN201910852597A CN110704180B CN 110704180 B CN110704180 B CN 110704180B CN 201910852597 A CN201910852597 A CN 201910852597A CN 110704180 B CN110704180 B CN 110704180B
- Authority
- CN
- China
- Prior art keywords
- unscheduled
- task
- tasks
- idle server
- workflow
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5072—Grid computing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5077—Logical partitioning of resources; Management or configuration of virtualized resources
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention provides a workflow scheduling method based on a hybrid cloud, which comprises the following steps: detecting whether an idle server set exists in the private cloud; when the existence is determined, screening a target idle server from the idle server set, screening an unscheduled task with the completion time less than the deadline time and the closest unscheduled task from the unscheduled task set, and scheduling the unscheduled task to the target idle server; and when the task is determined to be absent, screening the rented target virtual machine from the public cloud, screening the unscheduled task with the completion time less than the deadline time and the closest unscheduled task from the unscheduled task set, and scheduling the unscheduled task to the target virtual machine. According to the technical scheme, the task with the completion time closest to the deadline is dispatched to the local resource or the public resource with the highest cost performance, so that the task is not limited to an independent task on the premise of meeting the constraints of task dependency, workflow deadline and the like, the applicability is wider, and the utilization rate of private resources and the expenditure of public resources are greatly optimized.
Description
Technical Field
The invention relates to the technical field of cloud computing, in particular to a workflow scheduling method based on a hybrid cloud.
Background
The hybrid cloud integrates public cloud and private cloud, and is a main mode and development direction of cloud computing in recent years. In the using process, for safety consideration, an enterprise prefers to store data in a private cloud, but simultaneously hopes to obtain computing resources of a public cloud, under the condition, a mixed cloud is increasingly adopted, the mixed cloud effectively solves the problem of insufficient private cloud resources by using the elasticity of the public cloud resources, workflow scheduling reasonably maps workflow tasks to the mixed computing resources, and although the task execution performance and the mixed resource use efficiency are optimized to a certain degree by the existing workflow scheduling method facing the mixed cloud, some problems still exist: the method is only oriented to independent tasks, the universality is poor, and in order to simplify the evaluation of the tasks on the resource requirements, the tasks or the resources are assumed to be isomorphic, so that the scheduling method has a narrow application range.
Disclosure of Invention
Based on at least one of the technical problems, the invention provides a novel workflow scheduling method based on the hybrid cloud, which has a wide application range and greatly optimizes the use efficiency of the hybrid cloud resources.
In view of this, the present invention provides a new workflow scheduling method based on a hybrid cloud, including: detecting whether a private cloud has an idle server set comprising a plurality of idle servers; when the idle server set is determined to exist, screening a target idle server from the idle server set, and calculating the deadline of each unscheduled task in an unscheduled task set of a workflow when the target idle server executes and the completion time of each unscheduled task on the target idle server, wherein the target idle server is the idle server with the largest service capacity in the idle server set; selecting unscheduled tasks with completion time less than the cutoff time and closest to the cutoff time from the unscheduled task set, scheduling the unscheduled tasks to the target idle server, and deleting the unscheduled tasks from the unscheduled task set and adding the unscheduled tasks to the task set of the private cloud until the unscheduled task set is an empty set; when the idle server set does not exist, screening renting target virtual machines from a public cloud, and calculating the deadline of each unscheduled task in an unscheduled task set of a workflow when the target virtual machine executes and the completion time of each unscheduled task on the target virtual machine, wherein the target virtual machine is a virtual machine which can complete all independent unscheduled tasks and has the highest cost performance; and screening unscheduled tasks with completion time less than the cutoff time and closest to the cutoff time from the unscheduled task set, scheduling the unscheduled tasks to the target virtual machine, and deleting the unscheduled tasks from the unscheduled task set until the unscheduled task set is an empty set.
In the above technical solution, preferably, the method further includes: and when the idle server set exists, if an unscheduled task with the completion time less than the deadline time and the closest unscheduled task is not screened from the unscheduled task set, deleting the target idle server from the idle server set, and re-screening the idle server with the maximum service capability from the idle server set.
In any one of the above technical solutions, preferably, the method further includes: and when the idle server set does not exist, if the unscheduled tasks with the completion time less than the deadline time and the closest unscheduled tasks are not screened from the unscheduled task set, re-screening and renting all independent unscheduled tasks from the common cloud and the virtual machines with the highest cost performance.
Through the technical scheme, the task with the completion time closest to the deadline is dispatched to the local resource or the public resource with the highest cost performance, so that the task is not limited to an independent task on the premise of meeting the constraints of task dependency, workflow deadline and the like, the applicability is wider, and the utilization rate of private resources and the expenditure of public resources are greatly optimized.
Drawings
FIG. 1 shows a flow diagram of a hybrid cloud-based workflow scheduling method according to one embodiment of the invention;
fig. 2 shows a flowchart of a hybrid cloud-based workflow scheduling method according to another embodiment of the present invention.
Detailed Description
In order that the above objects, features and advantages of the present invention can be more clearly understood, a more particular description of the invention will be rendered by reference to the appended drawings. It should be noted that the embodiments and features of the embodiments of the present application may be combined with each other without conflict.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced in other ways than those specifically described herein, and therefore the scope of the present invention is not limited by the specific embodiments disclosed below.
The technical solution of the present invention is further explained below with reference to fig. 1 and 2:
as shown in fig. 1, the workflow scheduling method based on the hybrid cloud includes the following steps:
step S102, whether the private cloud has an idle server set comprising a plurality of idle servers is detected.
And step S104, when the idle server set is determined to exist, screening a target idle server from the idle server set, and calculating the deadline of each unscheduled task in the unscheduled task set of the workflow when the target idle server executes and the completion time of each unscheduled task on the target idle server, wherein the target idle server is the idle server with the largest service capacity in the idle server set.
And S106, selecting unscheduled tasks with the completion time less than the deadline time and the closest unscheduled tasks from the unscheduled task sets, scheduling the unscheduled tasks to a target idle server, deleting the unscheduled tasks from the unscheduled task sets and adding the unscheduled tasks to the task sets of the private cloud until the unscheduled task sets are empty sets.
And S108, screening and renting the target virtual machine from the public cloud when the idle server set does not exist, and calculating the deadline of each unscheduled task in the unscheduled task set of the workflow in the execution of the target virtual machine and the completion time of each unscheduled task on the target virtual machine, wherein the target virtual machine is the virtual machine which can complete all independent unscheduled tasks and has the highest cost performance.
And S110, selecting the unscheduled tasks with the completion time less than the deadline time and the closest unscheduled tasks from the unscheduled task sets, scheduling the unscheduled tasks to the target virtual machine, and deleting the unscheduled tasks from the unscheduled task sets until the unscheduled task sets are empty sets.
Further, still include: when the idle server set exists, if the unscheduled tasks with the completion time less than the deadline time and the closest unscheduled tasks in the unscheduled task set are not screened, deleting the target idle server from the idle server set, and re-screening the idle server with the maximum service capability from the idle server set.
Further, still include: and when no idle server set exists, if the unscheduled tasks which are closest to the unscheduled tasks and have the completion time less than the deadline time are not screened from the unscheduled task set, screening and renting all independent unscheduled tasks from the common cloud again, wherein the unscheduled tasks can be completed by the virtual machines with the highest cost performance.
This is further illustrated by fig. 2 below:
step 1: judging whether an idle server exists in the private cloud or not, if not, executing the step 5, otherwise, setting the set P of the idle servers as P ═ PjSelecting an idle server with the largest service capacity from the | j ═ 1.. multidot.m }, and recording the idle server as pmaxI.e. the target idle server, and remembering its service capability as cmaxNamely, the service capability of the server satisfies:
wherein p isjIs the jth idle server, cjIs pjService capability of cmaxIs pmaxThe service capability of (2).
Step 2: calculating the deadline of the unscheduled task of the workflow according to the following calculation formula:
wherein T is the set of all tasks of the workflow, TU is the set of unscheduled tasks, TiAnd tjI and j, respectively, of the not scheduled task, sjIs tjWorkload of diAnd djAre each tiAnd tjCutoff time of eijIs tiTransmits to t after completionjD is the deadline of the workflow, blRepresenting the bandwidth of data transmission inside the private cloud.
And step 3: calculate each unscheduled task at pmaxAnd selecting the task with the completion time less than the cut-off time and the closest completion time, and recording as tmaxThe completion time of the unscheduled task is calculated as follows:
where T is the set of all tasks of the workflow, TU is the set of unscheduled tasks, TiAnd tjI and j, respectively, of the not scheduled task, siIs tiWork amount of ftiAnd ftjAre each tiAnd tjCompletion time of (e)jiIs tjTransmits to t after completioniAmount of data of ftmaxTo be already scheduled to pmaxThe latest completion time of the task of (b)lRepresenting the bandwidth of data transmission inside the private cloud.
And 4, step 4: judging whether an unscheduled task t with the completion time less than the deadline and the closest completion time in the step 3 existsmaxIf not, p is addedmaxDelete from free server set P (P ═ P \ { Pmax}) and execute step 1, otherwise, the task tmaxDispatch to Server pmaxTo get tmaxDelete from a set of unscheduled tasks (TU \ t)max}) and add it to the set of tasks scheduled to the private cloud, (TS ═ TS utou { t } @maxAnd h), wherein the TS is a task set scheduled to the private cloud, and whether a task which is not scheduled exists is judged, if yes, the step 3 is executed, otherwise, the process is ended.
And 5: renting the virtual machine which can complete all independent unscheduled tasks and has the highest cost performance on the public cloud, and recording as the virtual machinevmaxI.e. the target virtual machine, and remembering its service capability as fmaxWhere there are no dependent unscheduled tasks tiThe following conditions are satisfied:
the kth virtual machine vkThe cost performance calculation formula is fk/ukWherein f iskAnd ukAre each vkService capacity and unit price.
Step 6: calculating the deadline of the unscheduled task of the workflow according to the following calculation formula:
wherein T is the set of all tasks of the workflow, TU is the set of unscheduled tasks, TiAnd tjI and j, respectively, unscheduled tasks, diAnd djAre each tiAnd tjCutoff time of eijIs tiTransmits to t after completionjD is the deadline of the workflow, bpRepresenting the bandwidth of the data transmission inside the public cloud.
And 7: calculating the value of each unscheduled task at vmaxAnd selecting the task with the completion time less than the cut-off time and the closest completion time, and recording as tmaxThe completion time of the unscheduled task is calculated as follows:
wherein T is a set of all tasks of the workflow, TU is a set of unscheduled tasks, TS is a set of tasks scheduled to the private cloud, TiAnd tjI and j, respectively, unscheduled tasks, ftiAnd ftjAre each tiAnd tjCompletion time of (e)jiIs tjTransmits to t after completioniAmount of data of ftmaxTo have been scheduled to vmaxThe latest completion time of the task of (b)pBandwidth representing data transmission inside public cloud, bbRepresenting the data transmission bandwidth between the private cloud and the public cloud.
And 8: judging whether an unscheduled task t meeting the condition that the completion time is less than the cutoff time and the completion time is closest in the step 6 exists or notmaxIf not, executing step 5, otherwise, executing the task tmaxDispatch to Server vmaxAnd will tmaxDelete from a set of unscheduled tasks (TU \ t)maxAnd h), judging whether a task which is not scheduled exists, if so, executing the step 7, otherwise, ending the process.
And scheduling the task with the completion time closest to the deadline to the local resource or the public resource with the highest cost performance, so as to achieve the purpose of optimizing the utilization rate of the private resource and the public resource expenditure on the premise of meeting the constraints of task dependency, workflow deadline and the like.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (1)
1. A workflow scheduling method based on a hybrid cloud is characterized by comprising the following steps:
detecting whether a private cloud has an idle server set comprising a plurality of idle servers;
when the idle server set exists, screening a target idle server from the idle server set, calculating the deadline of each unscheduled task in an unscheduled task set of the workflow when the target idle server executes and the completion time of each unscheduled task on the target idle server, and judging whether an idle server exists in the private cloud, wherein the idle server set P is { P { (P) }jSelecting an idle server with the largest service capacity from the | j ═ 1.. multidot.m }, and recording the idle server as pmaxI.e. the target idle server, and remembering its service capability as cmaxNamely, the service capability of the server satisfies:
wherein p isjIs the jth idle server, cjIs pjService capability of cmaxIs pmaxThe service capability of (2); calculating the deadline of the unscheduled task of the workflow according to the following calculation formula:
wherein T is the set of all tasks of the workflow, TU is the set of unscheduled tasks, TiAnd tjI and j, respectively, of the not scheduled task, sjIs tjWorkload of diAnd djAre each tiAnd tjCutoff time of eijIs tiTransmits to t after completionjD is the deadline of the workflow, blRepresenting the bandwidth of data transmission in the private cloud, and calculating the bandwidth of each unscheduled task in pmaxAnd selecting the task with the completion time less than the cut-off time and the closest completion time, and recording as tmaxThe completion time of the unscheduled task is calculated as follows:
where T is the set of all tasks of the workflow, TU is the set of unscheduled tasks, TiAnd tjI and j, respectively, of the not scheduled task, siIs tiWork amount of ftiAnd ftjAre each tiAnd tjCompletion time of (e)jiIs tjTransmits to t after completioniAmount of data of ftmaxTo be already scheduled to pmaxThe latest completion time of the task of (b)lRepresenting the bandwidth of data transmission inside a private cloud, wherein the target idle server is an idle server with the largest service capacity in the idle server set;
selecting unscheduled tasks with completion time less than the cutoff time and closest to the cutoff time from the unscheduled task set, scheduling the unscheduled tasks to the target idle server, and deleting the unscheduled tasks from the unscheduled task set and adding the unscheduled tasks to the task set of the private cloud until the unscheduled task set is an empty set;
when the idle server set does not exist, screening renting target virtual machines from the public cloud, calculating the deadline of each unscheduled task in the unscheduled task set of the workflow when the target virtual machine executes and the completion time of each unscheduled task on the target virtual machine, renting the virtual machine which can complete all independent unscheduled tasks and has the highest cost performance on the public cloud, and recording the virtual machine as vmaxI.e. the target virtual machine, and recordsIts service capability is fmaxWhere there are no dependent unscheduled tasks tiThe following conditions are satisfied:
the kth virtual machine vkThe cost performance calculation formula is fk/ukWherein f iskAnd ukAre each vkThe deadline of the unscheduled task of the workflow is calculated by the following formula:
where T is the set of all tasks of the workflow, TU is the set of unscheduled tasks, TiAnd tjI and j, respectively, unscheduled tasks, diAnd djAre each tiAnd tjCutoff time of eijIs tiTransmits to t after completionjD is the deadline of the workflow, bpThe bandwidth of data transmission in the public cloud is represented, and the completion time of the unscheduled task is calculated in the following mode:
where T is the set of all tasks of the workflow and TU is the set of unscheduled tasksTS is a set of tasks scheduled to the private cloud, tiAnd tjI and j, respectively, unscheduled tasks, ftiAnd ftjAre each tiAnd tjCompletion time of (e)jiIs tjTransmits to t after completioniAmount of data of ftmaxTo have been scheduled to vmaxThe latest completion time of the task of (b)pBandwidth representing data transmission inside public cloud, bbRepresenting data transmission bandwidth between a private cloud and a public cloud, wherein the target virtual machine is a virtual machine which can complete all independent tasks which are not scheduled and has the highest cost performance;
selecting an unscheduled task with the completion time less than the cutoff time and the closest unscheduled task from the unscheduled task set, scheduling the unscheduled task to the target virtual machine, and deleting the unscheduled task from the unscheduled task set until the unscheduled task set is an empty set; further comprising:
when the idle server set exists, if an unscheduled task with a completion time smaller than a deadline time and closest to the unscheduled task is unscheduled from the unscheduled task set, deleting the target idle server from the idle server set, and screening an idle server with the maximum service capability from the idle server set again; further comprising:
and when the idle server set does not exist, if the unscheduled tasks with the completion time less than the deadline time and the closest unscheduled tasks are not screened from the unscheduled task set, re-screening and renting all independent unscheduled tasks from the common cloud and the virtual machines with the highest cost performance.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910852597.5A CN110704180B (en) | 2019-09-10 | 2019-09-10 | Workflow scheduling method based on hybrid cloud |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910852597.5A CN110704180B (en) | 2019-09-10 | 2019-09-10 | Workflow scheduling method based on hybrid cloud |
Publications (2)
Publication Number | Publication Date |
---|---|
CN110704180A CN110704180A (en) | 2020-01-17 |
CN110704180B true CN110704180B (en) | 2021-04-06 |
Family
ID=69195160
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201910852597.5A Active CN110704180B (en) | 2019-09-10 | 2019-09-10 | Workflow scheduling method based on hybrid cloud |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN110704180B (en) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112406748B (en) * | 2020-11-19 | 2023-08-01 | 广州瑞修得信息科技有限公司 | Vehicle data acquisition method and device and vehicle diagnosis system |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108009023A (en) * | 2017-11-29 | 2018-05-08 | 武汉理工大学 | Method for scheduling task based on BP neural network time prediction in mixed cloud |
CN108021435A (en) * | 2017-12-14 | 2018-05-11 | 南京邮电大学 | A kind of cloud computing task stream scheduling method with fault-tolerant ability based on deadline |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9086923B2 (en) * | 2011-09-16 | 2015-07-21 | Rutgers, The State University Of New Jersey | Autonomic workflow management in dynamically federated, hybrid cloud infrastructures |
-
2019
- 2019-09-10 CN CN201910852597.5A patent/CN110704180B/en active Active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108009023A (en) * | 2017-11-29 | 2018-05-08 | 武汉理工大学 | Method for scheduling task based on BP neural network time prediction in mixed cloud |
CN108021435A (en) * | 2017-12-14 | 2018-05-11 | 南京邮电大学 | A kind of cloud computing task stream scheduling method with fault-tolerant ability based on deadline |
Non-Patent Citations (1)
Title |
---|
基于二次聚类的多目标混合云任务调度算法;李建丽等;《浙江大学学报》;20170630;摘要、第1-4节 * |
Also Published As
Publication number | Publication date |
---|---|
CN110704180A (en) | 2020-01-17 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11847103B2 (en) | Data migration using customizable database consolidation rules | |
CN101645022B (en) | Work scheduling management system and method for a plurality of colonies | |
CN107729139B (en) | Method and device for concurrently acquiring resources | |
US20200104377A1 (en) | Rules Based Scheduling and Migration of Databases Using Complexity and Weight | |
US20200104375A1 (en) | Data Migration Using Source Classification and Mapping | |
CA3000422C (en) | Workflow service using state transfer | |
CN113243005A (en) | Performance-based hardware emulation in on-demand network code execution systems | |
CN111796908B (en) | System and method for automatic elastic expansion and contraction of resources and cloud platform | |
CN103092698A (en) | System and method of cloud computing application automatic deployment | |
CN103902646A (en) | Distributed task managing system and method | |
CN112162865A (en) | Server scheduling method and device and server | |
CN102567072A (en) | Resource allocation method, resource allocation device and resource allocation system | |
CN104050042A (en) | Resource allocation method and resource allocation device for ETL (Extraction-Transformation-Loading) jobs | |
US8874787B2 (en) | Optimized consumption of third-party web services in a composite service | |
US20210406053A1 (en) | Rightsizing virtual machine deployments in a cloud computing environment | |
CN113228574A (en) | Computing resource scheduling method, scheduler, internet of things system and computer readable medium | |
CN102457578A (en) | Distributed network monitoring method based on event mechanism | |
CN114153580A (en) | Cross-multi-cluster work scheduling method and device | |
US20100274621A1 (en) | Method and System for Integration of Systems Management with Project and Portfolio Management | |
CN110704180B (en) | Workflow scheduling method based on hybrid cloud | |
WO2017074320A1 (en) | Service scaling for batch processing | |
CN114116157A (en) | Multi-edge cluster cloud structure in edge environment and load balancing scheduling method | |
CN110084507A (en) | The scientific workflow method for optimizing scheduling of perception is classified under cloud computing environment | |
CN105446812A (en) | Multitask scheduling configuration method | |
CN107797870A (en) | A kind of cloud computing data resource dispatching method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |