CN104536806A - Workflow application flexible resource supplying method in cloud environment - Google Patents

Abstract

The invention discloses a workflow application flexible resource supplying method in a cloud environment. A demand-based charging model based on a time interval, data transmission time, virtual machine starting time and software mounting (downloading) time are considered. The multi-rules based heuristic (MRH) method mainly includes the steps that A, deadlines of sub-tasks are divided, and the step A is mainly characterized in that through removing the constraint from the original problem that charging is performed according to the time interval, the original problem is simplified, then the integer planning model of the problem is solved, and therefore the deadlines of all the sub-tasks are obtained; B, according to a multi-rules based sub-task and virtual machine time slot matching method, the step B is mainly characterized in that three priority rules considering multiple factors are provided, and the tasks are matched with suitable virtual machine available time slots. Through the appropriate sub-task deadline division and the multi-rules based time slot matching method, the resource rent cost for a whole workflow application is greatly lowered.

Description

Workflow application flexible resource Supply Method under a kind of cloud environment

Technical field

The present invention relates to the workflow application flexible resource Supply Method under a kind of cloud environment, belong to cloud computing resources dispatching technique field.

Background technology

Based on the development of Distributed Calculation particularly grid computing, create a kind of new services computation model: cloud computing (Cloud Computing).Cloud computing is a kind of pattern can being accessed a configurable computing resource sharing pond by network in mode easily, as required, this resource sharing pond with minimum administration overhead and minimum mutual with supplier, can configure rapidly, provides or releasing resource.The main advantage of cloud computing is: promptly can reduce hardware cost and promote computing power and memory capacity etc.; User can drop into extremely low cost and obtain high calculating quality, and need not reinvest the hardware device buying costliness, maintains frequently and upgrades.

Due to above-mentioned advantage, many companies and research institution all attempt its real-time analysis, online advertisement and scientific algorithm application etc. to move in existing business cloud, to solve data processing problem when data volume increases on a large scale.Realize calculate or store tasks and the reasonable mapping bought between resource be huge challenge for user.Cloud provide resource provisioning scheme more flexibly from convergent-divergent character (scale of construction according to application obtains at any time or discharges the resource of suitable type and quantity, according to by the situation charging used).Too low resource distribution can cause the decline of system performance, causes the delay of off period; And too high resource distribution can cause idle time slot, add unnecessary expense.Therefore, user needs to propose a kind of rational resource from convergent-divergent strategy to save rent expense under the prerequisite ensureing system performance.

Analysis and scientific algorithm application are the class important application based on business cloud, usually need to solve a large amount of data.In order to reduce the execution time, can divide data, solving with parallel task.Meanwhile, these application are also all transmitted by data between multiple sequential steps and parallel and serial task and are formed.This kind of application available work stream walked abreast with serial task that comprises carries out modeling.Meanwhile, these application have the off period mostly.Therefore, rent balance between expense and workflow completion date to seek resource, cloud user needs VM example workflow task being assigned to applicable type and number to get on.

But, due in actual applications, renting of cloud environment adopts pattern hourly usually, and needs to consider the factors such as software preparation time, data transmission period, the network structure of complexity and the diversity of task, and resource provisioning and scheduling problem are just difficult to solve.

At present existing method and project are absorbed in the workflow schedule problem solving complicated factor under cloud environment.Dynamic conditioning strategy but these methods are not considered to happen suddenly in cloud computing environment under demand and limited case, also seldom consider in practical application the restraining factors such as large-scale data transmission time, software unit switching time, therefore can not apply in the background discussed with the present invention.

Summary of the invention

Goal of the invention: for problems of the prior art with not enough, the invention provides the workflow application flexible resource Supply Method under a kind of cloud environment, to reduce virtual machine rent under the prerequisite ensureing system performance.

Technical scheme: the workflow application flexible resource Supply Method under a kind of cloud environment, comprises the following steps:

A. workflow divided by the phase: the execution time of each task in calculation workflow in each virtual machine type and cost, to minimize the total hiring cost of virtual machine for target not violating under workflow off period prerequisite, do not consider the constraint of interval charging, workflow issues is reduced to integer programming problem, and solve the virtual machine type obtaining each task choosing, determine task execution time; According to first order relation and task execution time between workflow off period, task, determine each task deadline;

B. task scheduling is carried out according to priority rule: according to the ascending scheduler task successively of the degree of depth of task in workflow, for each task according to its off period, based on priority rule, calculate the priority of all virtual machine time slots, by task matching to the virtual machine time slot with limit priority, it is preferential that described priority rule comprises minimum fresh tenancy time interval, and minimum total process cost is preferential, and the task processing time of the best and the matching degree of time slot length preferential.

In described steps A, the execution time of each task in each virtual machine type comprises data transmission period, virtual machine and software preparation time.

According to first order relation and task execution time between workflow off period, task in described steps A, determine that the concrete steps of each task deadline comprise:

S1. to task v each in set of tasks V, will be initialized as data transmission period, VM setup time, software preparation time and task execution time sum, calculate the earliest start time EST of each task _v, earliest finish time EFT _vend time LFT the latest _v;

S2. critical path generation method grey iterative generation initial key path CP is adopted, CP _[i]represent i-th task of CP;

S3. judge whether CP is empty, if it is empty, goes to step S10; Otherwise, go to step S4;

S4. the T.T. calculating CP floats $T_{\mathrm{CP}}^{\mathrm{float}}={\mathrm{\Σ}}_{{\mathrm{CP}}_{\left[k\right]\∈{\mathrm{CP}}^{\′}}}({\mathrm{EST}}_{{\mathrm{CP}}_{[k+1]}}-{\mathrm{EFT}}_{{\mathrm{CP}}_{\left[k\right]}})+{\mathrm{LFT}}_{{\mathrm{CP}}_{\left[l\right]}}-$ ${\mathrm{EFT}}_{{\mathrm{CP}}_{\left[l\right]}};$ Wherein, CP '=CP/V ^fix/ { CP _[l]; , l is the length of CP.

S5. judge whether be greater than 0, if be greater than 0, go to step S6; Otherwise, go to step S12;

S6. initialization i=1;

S7. judge i≤l, if so, go to step S8; Otherwise, go to step S11;

S8. to the task CP in CP _[i], judge CP _[i]whether belong to pinned task set V ^fix, wherein pinned task set V ^fixmeet for the time floats the set of all task v composition; If belong to, go to step S10;

S9. CP is calculated _[i]what be assigned with floats $T_{C{P}_{\left[i\right]}}^{\mathrm{dis}}=T_{\mathrm{CP}}^{\mathrm{float}}\×T_{{\mathrm{CP}}_{\left[i\right]}}^{\mathrm{float}}/{\mathrm{\Σ}}_{v_k\∈\mathrm{CP}/V^{\mathrm{fix}}}\left\{T_{v_k}^{\mathrm{float}}\right\},$ Upgrade its time floating upgrade early start and the earliest finish time of its subsequent tasks;

S10. i=i+1 is upgraded;

S11. end time the latest and the V of all tasks in V is upgraded ^fix, go to step S4;

S12. generate next critical path CP, go to step S3;

S13., the task deadline of each task is set for

In described step B, comprising:

B1. the off-the-shelf task-set θ of initialization is the set { v only comprising beginning task ₀, calculate the earliest start time of each task in θ;

B2. dividing θ according to the degree of depth of task each in off-the-shelf task-set θ is each subset, the subset θ that selected depth is minimum _s, according to the execution time, non-ascending sort is carried out to task wherein, chooses wherein first task v _tas the task of being about to carry out dispatching; If v _tfor sky, go to step B5;

B3. for current task v _t, an example of the virtual machine type chosen temporarily is added in the virtual machine instance set I rented in described cloud environment; According to earliest start time and the off period of task, draw v _tcurrent available time slot set

B4. according to priority rule, task v is calculated _t? in priority value on each time slot slot, comprise and obtaining according to minimum fresh tenancy time interval priority rule obtain according to minimum total process cost priority rule and obtain according to the task processing time of the best and the matching degree priority rule of time slot length by inciting somebody to action with integrate by setting weight, obtain v _tfinal priority value on slot, chooses the time slot slot wherein with limit priority _s, by task v _tdistribute to slot _s; Remove untapped VM example in I; Upgrade off-the-shelf task-set θ, if θ is not empty, go to step B2;

B5. task is mated with virtual machine time slot and is terminated, and exits.

Beneficial effect: compared with prior art, former problem reduction is DTCTP (Discrete Time CostTradeoff Problem) problem by the present invention, utilizes integer programming model (IP) to solve the virtual machine type of each task; After sub-off period of task determines, according to the priority rule of three considerations proposed multifactor (as rented the match condition etc. of the reusing of interval, VM load time, software preparation time, data transmission period, execution efficiency, time slot and task execution time length), for virtual machine (VM) time slot that each task choosing is suitable, task is divided one by one and tasks VM example, and ensure not violate its sub-off period; All task assignments complete, then obtain the scheduling result that can be used for actual execution.Heuristic Multiple-Rules based Heuristc (MRH) resource provisioning method based on more rules provided by the invention, consider based on the charging model in the time interval, data transmission period, VM start-up time and software preparation time etc. situation the scheduling problem of workflow in cloud environment, the software reuse rate that maximum resource is reused rate and filled.

Accompanying drawing explanation

Fig. 1 is the structural drawing that the embodiment of the present invention realizes the workflow application flexible resource Supply Method under cloud environment;

Fig. 2 is the process flow diagram of workflow off period division part in the embodiment of the present invention;

Fig. 3 is the process flow diagram carrying out task scheduling part in the embodiment of the present invention according to priority rule.

Embodiment

Below in conjunction with specific embodiment, illustrate the present invention further, these embodiments should be understood only be not used in for illustration of the present invention and limit the scope of the invention, after having read the present invention, the amendment of those skilled in the art to the various equivalent form of value of the present invention has all fallen within the application's claims limited range.

The embodiment of the present invention realizes the structural drawing of the workflow application flexible resource Supply Method under cloud environment as shown in Figure 1, the data center 15 comprising cloud service provider 11, workflow application 12, elasticity intermediary 13, workflow schedule device 14 and be made up of the virtual machine leased.Suppose in the present embodiment in described cloud environment, mainly to comprise two kinds of roles, i.e. cloud service supplier (such as IaaS supplier) and cloud service user.Cloud service supplier provides resource lease service (such as virtual machine and storage resources) to cloud service user, and resource lease is according to time interval charging.Cloud service user, by the resource of lease cloud service supplier, sets up the virtual data center of oneself to serve the workflow application of oneself.Elasticity intermediary and workflow schedule device provide automatic scale Regulation mechanism for cloud service user, help user's minimized resource hiring cost under the prerequisite not violating the workflow application off period.

Workflow G={V, E} are directed acyclic graph, wherein a V={v ₀, v ₁..., v _n+1the set comprising N+2 task, E={ (i, j) ∨ i < j} is the first order constrained between each task, i.e. v _jcan not at v _istart before completing, P _iand S _ibe respectively task v _idirect precursor collection and immediate successor collection; D represents the workflow off period.

When workflow G={V, an E} submit to system, workflow packages being contained of task V={v is responsible for by workflow schedule device ₀, v ₁..., v _n+1mate with virtual machine instance time slot slot in most suitable mode, namely under the prerequisite reaching the requirement of workflow off period, minimize hiring cost.The resource of cloud service suppliers is leased or discharged in elasticity intermediary, according to the scheduling result of workflow schedule device.The scheduling of workflow schedule device is divided into the workflow off period divide and carry out task scheduling two parts according to priority rule.

Fig. 2 is the process flow diagram of workflow off period division part in the embodiment of the present invention.As shown in Figure 2, the concrete steps of workflow off period division are as follows:

Step s201, for former problem, assuming that the task processing time all comprise the setup time of data transmission period and virtual machine and software, and virtual machine instance (instead of discrete way) charging in a continuous manner.First in the execution time that we add data transmission period and virtual machine and software preparation time to each mode; Accordingly, calculation task v _iat virtual machine type δ _ton execution time p _{i, t}with Executing Cost c _{i, t}.The constraint of former problem according to interval charging deleted, this problem can be reduced to type decision problem, with integer programming (IP) modeling.

Step s202, adopt integer programming solver, such as ask approximate optimal solution by the problem of CPLEX to this simplification, gained solution is R; In R, give the virtual machine type of the selection of each subtask, namely perform mode.Wherein represent task v in R _ivirtual machine type sequence number.

Step s202, asks approximate optimal solution by the problem of CPLEX to this simplification, and gained solution is R, wherein represent task v in R _ivirtual machine type sequence number.

Step s203, to task v each in V, be initialized as data transmission period, VM setup time, software preparation time and task execution time sum, calculate the earliest start time EST of each task _v, earliest finish time EFT _vend time LFT the latest _v.

Step s204, adopts the critical path generation method grey iterative generation initial key path CP that Abrishami etc. proposes, CP _[i]represent i-th task of CP.

Step s205, judges whether CP is empty, if it is empty, goes to step s212; Otherwise, go to step s206.

Step s206, the T.T. calculating CP floats $T_{\mathrm{CP}}^{\mathrm{float}}={\mathrm{\&Sigma;}}_{{\mathrm{CP}}_{\left[k\right]\&Element;{\mathrm{CP}}^{\&prime;}}}({\mathrm{EST}}_{C{P}_{[k+1]}}-{\mathrm{EFT}}_{{\mathrm{CP}}_{\left[k\right]}})+$ ${\mathrm{LFT}}_{{\mathrm{CP}}_{\left[l\right]}}-{\mathrm{EFT}}_{{\mathrm{CP}}_{\left[l\right]}}.$ Wherein, CP '=CP/V ^fix/ { CP _[l], l is the length of CP.

Step s207, judges whether be greater than 0, if be greater than 0, go to step s208; Otherwise, go to step s214.

Step s208, initialization i=1.

Step s209, judges i≤l, if so, goes to step s210; Otherwise, go to step s213.

Step s210, to the task CP in CP _[i], judge CP _[i]whether belong to pinned task set V ^fix, wherein pinned task set V ^fixmeet for the time floats the set of all task v composition; If belong to, go to step s212;

Step s211, calculates CP _[i]what be assigned with floats $T_{C{P}_{\left[i\right]}}^{\mathrm{dis}}=T_{\mathrm{CP}}^{\mathrm{float}}\&times;T_{{\mathrm{CP}}_{\left[i\right]}}^{\mathrm{float}}/{\mathrm{\&Sigma;}}_{v_k\&Element;\mathrm{CP}/V^{\mathrm{fix}}}\left\{T_{v_k}^{\mathrm{float}}\right\},$ Upgrade its time floating upgrade early start and the earliest finish time of its subsequent tasks.

Step s212, upgrades i=i+1.

Step s213, upgrades end time the latest and the V of all tasks in V ^fix, go to step s206.

Step s214, generates next critical path CP, goes to step s205.

Step s215, arranges the task deadline of each task for go to step s301.

Fig. 3 is the process flow diagram carrying out task scheduling part in the embodiment of the present invention according to priority rule.As shown in Figure 3, the concrete steps of task scheduling are carried out according to priority rule as follows:

Step s301, the off-the-shelf task-set θ of initialization are the set { v only comprising beginning task ₀, calculate the earliest start time of each task in θ.

Step s302, dividing θ according to the degree of depth of task each in off-the-shelf task-set θ is each subset.

Step s303, the subset θ that selected depth is minimum _s, according to the execution time, non-ascending sort is carried out to task wherein, chooses wherein first task v _tas the task of being about to carry out dispatching.

Step s304, judges v _twhether be empty, if v _tfor sky, then method terminates; Otherwise, go to step s305.

Step s305, for v _tchoose the VM type of mode, choose an example and be temporarily added in the VM example collection I rented in described cloud environment.

Step s306, according to earliest start time and subtask off period, draws v _tcurrent available time slot set

Step s307 is right in each time slot slot, calculate its priority value according to three priority rules and choose in have minimum (namely priority is the highest) time slot slot _s.Wherein three priority rules calculate and are respectively:

1) minimum fresh tenancy time interval is preferential

Task v _tbe assigned to the number causing on the number virtual machine of fresh tenancy time interval number between required message area and actual fresh tenancy timeslice, draw rule value circular is: 1) calculate v _tnumber between message area required on slot type Virtual machine, adds 1, obtains required maximum interval number; 2) by v _tthe number of the actual fresh tenancy timeslice that slot causes is divided by step 1) in the maximum interval number that obtains, thus to obtain

2) minimum total process cost is preferential

Calculation task v _tduring the priority value of this rule on slot, first according to slot place virtual machine type, draw v _ttask execution time; Then, check it is whether the virtual machine of fresh tenancy, if so, then the processing time will add the virtual machine load time; If the virtual machine instance had, then check whether slot has v _tthe software, the control that need, if do not had, then need the download of software and set-up time to be added on task execution time.Finally, calculate all preorder task datas and be transferred to T.T. on this slot, add task execution time to; Finally draw v _ttotal processing time on this slot.By this value normalization, draw second regular priority value

3) the best task processing time and the matching degree of time slot length preferential

Definition: task v _tmatching degree on slot is v _tbetween the message area that the time after slot that is assigned to uses, the length of two ends rest segment is divided by the charging burst length of twice.The priority value of this rule is circular is: 1) obtain v _tshared set continuously between message area on slot; 2) v is calculated _ttwo ends waste sum between shared message area; 3) by wasting sum divided by the charging burst length of twice, obtain

By inciting somebody to action with integrate (such as 100: 10: 1) by certain weight, obtain v _tfinal priority value on slot.

Step s308, by task v _tdistribute to slot _s; Remove untapped VM example in I.

Step s309, predecessor task being all scheduled of task adds θ to upgrade; Go to step s302.

By said process, the present invention realizes the supply of cloud environment resource dynamic, when effectively ensure that system performance, minimizes the hiring cost of system total resources.Although illustrate and describe embodiments of the invention, those having ordinary skill in the art will appreciate that: can carry out multiple change, amendment, replacement and modification to these embodiments when not departing from principle of the present invention and aim, scope of the present invention is by claim and equivalents thereof.

Claims (7)

1. the workflow application flexible resource Supply Method under cloud environment, it is characterized in that, the method comprises the following steps:

A. workflow divided by the phase: the execution time of each task in calculation workflow in each virtual machine type and cost, to minimize the total hiring cost of virtual machine for target not violating under workflow off period prerequisite, do not consider the constraint of interval charging, workflow issues is reduced to integer programming problem, and solve the virtual machine type obtaining each task choosing, determine task execution time; According to first order relation and task execution time between workflow off period, task, determine each task deadline;

B. task scheduling is carried out according to priority rule: according to the ascending scheduler task successively of the degree of depth of task in workflow, for each task according to its off period, based on priority rule, calculate the priority of all virtual machine time slots, by task matching to the virtual machine time slot with limit priority, it is preferential that described priority rule comprises minimum fresh tenancy time interval, and minimum total process cost is preferential, and the task processing time of the best and the matching degree of time slot length preferential.

2. the workflow application flexible resource Supply Method under cloud environment as claimed in claim 1, it is characterized in that, in described steps A, the execution time of each task in each virtual machine type comprises data transmission period, virtual machine and software preparation time.

3. the workflow application flexible resource Supply Method under cloud environment as claimed in claim 1, is characterized in that, according to first order relation and task execution time between workflow off period, task in described steps A, determines that the concrete steps of each task deadline comprise:

S1. to task v each in set of tasks V, will be initialized as data transmission period, VM setup time, software preparation time and task execution time sum, calculate the earliest start time EST of each task _v, earliest finish time EFT _vend time LFT the latest _v;

S2. critical path generation method grey iterative generation initial key path CP is adopted, CP _[i]represent i-th task of CP;

S3. judge whether CP is empty, if it is empty, goes to step S10; Otherwise, go to step S4;

S4. the T.T. calculating CP floats $T_{\mathrm{CP}}^{\mathrm{float}}={\mathrm{\&Sigma;}}_{{\mathrm{CP}}_{\left[k\right]\&Element;{\mathrm{CP}}^{\&prime;}}}({\mathrm{EST}}_{{\mathrm{CP}}_{[k+1]}}-{\mathrm{EFT}}_{{\mathrm{CP}}_{\left[k\right]}})+{\mathrm{LFT}}_{{\mathrm{CP}}_{\left[l\right]}}-$ wherein, CP '=CP/V ^fix/ { CP _[l]; , l is the length of CP;

S5. judge whether be greater than 0, if be greater than 0, go to step S6; Otherwise, go to step S12;

S6. initialization i=1;

S7. judge i≤l, if so, go to step S8; Otherwise, go to step S11;

S8. to the task CP in CP _[C], judge CP _[i]whether belong to pinned task set V ^fix, wherein pinned task set V ^fixmeet for the time floats the set of all task C composition; If belong to, go to step S10;

S9. CP is calculated _[i]what be assigned with floats $T_{{\mathrm{CP}}_{\left[i\right]}}^{\mathrm{dis}}=T_{\mathrm{CP}}^{\mathrm{float}}\&times;T_{{\mathrm{CP}}_{\left[i\right]}}^{\mathrm{float}}\&times;T_{{\mathrm{CP}}_{\left[i\right]}}^{\mathrm{float}}/{\mathrm{\&Sigma;}}_{v_k\&Element;\mathrm{CP}/V}\mathrm{fix}\left\{T_{v_k}^{\mathrm{float}}\right\},$ Upgrade its time floating upgrade early start and the earliest finish time of its subsequent tasks;

S10. i=i+1 is upgraded;

S11. end time the latest and the V of all tasks in V is upgraded ^fix, go to step S4;

S12. generate next critical path CP, go to step S3;

S13., the task deadline of each task is set for

4. the workflow application flexible resource Supply Method under cloud environment as claimed in claim 1, is characterized in that, in described step B, comprising:

B1. the off-the-shelf task-set θ of initialization is the set { v only comprising beginning task ₀, calculate in the earliest start time of each task;

B2. dividing θ according to the degree of depth of task each in off-the-shelf task-set θ is each subset, the subset θ that selected depth is minimum _s, according to the execution time, non-ascending sort is carried out to task wherein, chooses wherein first task L _tas the task of being about to carry out dispatching; If v _tfor sky, go to step B5;

B3. for current task v _t, an example of the virtual machine type chosen temporarily is added in the virtual machine instance set I rented in described cloud environment; According to earliest start time and the off period of task, draw v _tcurrent available time slot set

B4. according to priority rule, task v is calculated _t? in priority value on each time slot slot, comprise and obtaining according to minimum fresh tenancy time interval priority rule obtain according to minimum total process cost priority rule and obtain according to the task processing time of the best and the matching degree priority rule of time slot length by inciting somebody to action with integrate by setting weight, obtain v _tfinal priority value on slot, chooses the time slot slot wherein with limit priority _s, by task v _tdistribute to slot _s; Remove untapped VM example in I; Upgrade off-the-shelf task-set θ, if θ is not empty, go to step B2;

B5. task is mated with virtual machine time slot and is terminated, and exits.

5. the workflow application flexible resource Supply Method under cloud environment as claimed in claim 4, is characterized in that, described in computing method be: 1) calculate v _tnumber between message area required on slot type Virtual machine, adds 1, obtains required maximum interval number; 2) by v _tthe number of the actual fresh tenancy timeslice that slot causes is divided by step 1) in the maximum interval number that obtains, thus to obtain

6. the workflow application flexible resource Supply Method under cloud environment as claimed in claim 4, is characterized in that, described in computing method be:

First, according to slot place virtual machine type, v is drawn _ttask execution time;

Then, check it is whether the virtual machine of fresh tenancy, if so, then the virtual machine load time is added on task execution time; If the virtual machine instance had, then check whether slot has v _tthe software needed, if do not had, then adds on task execution time by the download of software and set-up time;

Finally, calculate all preorder task datas and be transferred to T.T. on this slot, and add on task execution time, draw v _ttotal processing time on this slot, obtain after normalization

7. the workflow application flexible resource Supply Method under cloud environment as claimed in claim 4, is characterized in that, described in by calculating v _tafter being assigned to slot, the time of task two sections waste obtains, and circular is: 1) obtain v _tshared set continuously between message area on slot; 2) v is calculated _ttwo ends waste sum between shared message area; 3) by wasting sum divided by the charging burst length of twice, obtain