CN104021040A - Cloud computing associated task scheduling method and device based on time constraint - Google Patents

Abstract

The present invention discloses a cloud computing associated task scheduling method and device based on time constraints. The method includes the following steps. First, the method of constructing a DAG graph is used to represent the associated relationship between tasks in the associated task group; and then , using the idea of layering the DAG graph, dividing non-associated tasks in the same layer into the same task set; then, virtualizing hardware resources, establishing a virtual machine cluster to provide a place for task execution; finally, using time constraints The hierarchical scheduling method schedules the tasks in the task set of each layer to the most suitable virtual machine for execution to ensure that the tasks are completed on time. The technical solution adopted by the present invention effectively solves the problem of execution delay in the scheduling process of cloud computing-related tasks in the prior art, can minimize the occurrence of task delays, and ensure that tasks are completed within the time expected by the user. Effective utilization of virtual machine resources.

Description

Method and device for scheduling associated tasks in cloud computing based on time constraints

technical field

The present invention relates to a method and device for scheduling associated tasks of cloud computing, in particular to a method and device for scheduling associated tasks of cloud computing based on time constraints.

Background technique

Cloud computing technology is the focus and frontier field of computer technology research at home and abroad. Cloud computing provides dynamically scalable virtual resources to users in the form of services, which can effectively reduce user hardware expenses and obtain high-quality resources at a low cost. quality of service.

The development of virtualization technology provides a good solution for cloud computing. By virtualizing hardware resources, multiple independent virtual machine clusters are formed. Users can use virtual machines by applying, which can effectively save the cost of hardware resources. Improve the effective utilization of resources and save management costs. The main function of cloud computing task scheduling is to assign the task application submitted by the user to the virtual machine through a certain method, and the virtual machine completes the task submitted by the user.

Among the tasks submitted by the user, some tasks are not single independent tasks, but associated tasks with a sequence of execution. These associated tasks cannot be assigned to the virtual machine at the same time during the task scheduling process, and need to wait for their predecessors. Tasks can only be scheduled after they are completed. In the process of scheduling associated tasks, once the execution of a certain task is delayed, it will affect the execution of subsequent tasks, thereby affecting the completion time of the entire associated task group. At present, most of the existing associated task scheduling methods only consider the problem of reducing the communication cost between tasks, but there are not many solutions for the situation of task delay.

When the user submits a group of related tasks, how to minimize the delay of the task and ensure that the task can be completed within the time expected by the user, and find more parallel tasks to be scheduled to the appropriate virtual machine to achieve the effective use of resources becomes The primary problem to be solved by the present invention.

Contents of the invention

Purpose of the invention: The purpose of the present invention is to solve the problem of task delay in the scheduling process of related tasks. In the task scheduling process, since each task is related to each other in the order of execution, that is, all tasks have predecessors or successors. Tasks, when a task is delayed during execution, the execution of subsequent tasks must be affected. How to maximize the elimination of this impact and make effective use of virtual machine resources is an important problem to be solved by the present invention.

Technical solution: a method for scheduling associated tasks in cloud computing based on time constraints, including the following steps:

(1) Establish a DAG graph (directed graph without loops) for all tasks in the associated task group according to the order in which they are executed;

(2) divide all tasks into different task sets according to the DAG diagram constructed in the step (1), and limit the time constraints for each layer of task sets;

(3) Virtualize hardware resources and build virtual machine clusters;

(4) Scheduling tasks to corresponding virtual machines in the virtual machine cluster according to the time constraints of each layer of task sets.

In the associated tasks described in the present invention, the relevance between tasks is reflected in the execution order of the tasks, that is, each task has its predecessor or successor task, and only when the direct predecessor task of each task is completed, its successor task to start executing. The specific process of establishing DAG graph in described step (1) is:

(1.1) Define G={T,Λ,C} as a triplet describing the associated task group, where T is the set of all tasks in the associated task graph, Λ is the set of execution order between tasks, and C is the task A collection of communication overheads between tasks (the data required for each task before execution is known, and the communication overhead refers to the ratio of the amount of data transmission between tasks to the system network bandwidth), when two tasks with a direct relationship are connected When assigned to execute on the same virtual machine, there is no need for data transmission between tasks, so the communication overhead is 0. When two directly related tasks are assigned to different virtual machines for execution, the communication overhead can be calculated by the following formula :

$\mathrm{<span\; class="notranslate">\; C</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; data</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; transmission</span>}}{\mathrm{<span\; class="notranslate">\; BW</span>}}$

Among them, data_transmission indicates the amount of data transmission between tasks, and BW refers to the system network bandwidth.

(1.2) Obtain the DAG diagram of the associated task group according to the execution order of the tasks in the set Λ.

The hierarchical division of associated task groups in the step (2) mainly adopts the idea of layering the DAG graph, and divides the tasks at the same layer into different task sets. Different task sets at the same level are considered non-associated. It can be scheduled to be executed in the virtual machine at the same time. The specific process of task set division is as follows:

(2.1) layering the DAG graph according to the order from the entry task to the exit task;

(2.2) Divide tasks in the DAG diagram into task sets of corresponding layers according to their corresponding levels;

(2.3) Take the completion time of the associated task group expected by the user as the total time constraint of associated task scheduling, and calculate the time constraint of each layer task set according to the ratio of the number of tasks in each layer task set to the total number of tasks.

The construction of the virtual machine cluster in the step (3) is mainly to deploy multiple virtual machines on multiple physical servers to form a virtual machine cluster. The virtual machine integrates the operating system, application software and data required to complete the task, and the hardware virtualization method can effectively improve the utilization of physical resources. The specific steps of virtual machine cluster construction are as follows:

(3.1) Virtualize physical resources (memory resources, network resources, storage resources, CPU resources) and build multiple virtual machines;

(3.2) Deploy the operating system, applications and data required for user task execution in the virtual machine;

(3.3) Add the constructed virtual machine to the virtual machine cluster and wait for the task to be scheduled for execution.

The task scheduling in the step (4) is based on time constraints, that is, the user has time constraints on the completion of a group of associated tasks, and all tasks must be scheduled to be executed on a suitable virtual machine as much as possible and the tasks can Finished on time. The specific process of task scheduling is as follows:

(4.1) According to the associated task group that has divided the task set in the step (2), receive a layer of task sets hierarchically, assign the tasks in the task set to different idle virtual machines in the virtual machine cluster, and go to step (4.2 );

(4.2) Judging whether all the tasks in the received task set have been executed, if not, continue to wait for the tasks to be executed, otherwise go to step (4.3);

(4.3) Calculate the total completion time of all tasks in the received task set, and judge whether it is less than the constraint time of the task at this layer. If the actual completion time is less than the constraint time, go to step (4.1), otherwise go to step (4.4) ;

(4.4) Assign the tasks in the task set of the next layer to the same virtual machine as its predecessor tasks, and go to step (4.2).

In the process of task scheduling, the tasks in the task set of each layer can be scheduled to the virtual machine at the same time, and each task is assigned to a different virtual machine for execution as much as possible. Although at the expense of the communication overhead of the task, it can Effectively avoid the effect of task waiting caused by multiple tasks being assigned to the same virtual machine. After the task of each layer is completed, if the total time of the task of this layer is greater than the constraint time of the task, when scheduling the task of the next layer, assign the tasks in the task set of this layer to the same virtual machine as its direct predecessor task, This can effectively avoid the communication time between tasks caused by the assignment of the predecessor and successor tasks to different virtual machines. This method is used to make up for the delay caused by the previous layer of tasks.

A cloud computing associated task scheduling device based on time constraints, comprising: a task graph construction unit, a task layering unit, a virtual machine management unit and a task scheduling unit;

The task graph construction unit is used to receive associated task groups, and build a DAG graph for all tasks according to the sequence of task execution;

The task layering unit is connected to the task graph construction unit, including a task set division unit and a time constraint calculation unit, and the task set division unit is used to divide tasks into different task sets according to the hierarchy according to the DAG diagram constructed by the task construction unit, The time constraint calculation unit allocates the time constraints of the associated task group to the time constraints of each specific layer;

The virtual machine management unit is used to manage the virtual machine cluster, and can monitor the execution of tasks on the virtual machine;

The task scheduling unit is respectively connected with the task layering unit and the virtual machine management unit, and is used to allocate the layered task sets in the task layering unit to appropriate virtual machines managed by the virtual machine management unit according to time constraints.

The task scheduling unit includes a task set receiving unit, a task set allocation unit, a task set execution monitoring unit and a task set time calculation unit;

The task set receiving unit receives the task set to be executed and the time constraints of the task set from the task layering unit; the task set allocation unit allocates the tasks in the task set to virtual machines, and the task set execution monitoring unit monitors the execution of tasks , the task set time calculation unit calculates the completion time of all tasks in the task set and compares it with the constraint time of the task set, and the task set allocation unit determines the allocation of the task set in the next layer according to the comparison result.

The present invention adopts the above-mentioned technical scheme, and has the following beneficial effects: the present invention uses the method of constructing a DAG graph to represent the association relationship between tasks in the associated task group, and adopts the idea of layering the DAG graph, and the tasks in the same layer Non-related tasks are divided into the same task set, and a virtual machine cluster is established to provide a place for task execution. In the process of task scheduling, a hierarchical scheduling method based on time constraints is used to schedule tasks in each layer to the most suitable virtual environment. Execution in the computer effectively solves the problem of execution delay in the scheduling process of cloud computing-related tasks in the prior art, minimizes the occurrence of task delays, ensures that tasks can be completed within the time expected by users, and realizes virtual efficient use of machine resources.

Description of drawings

Fig. 1 is the overall frame diagram of the method embodiment of the present invention;

Fig. 2 is the DAG diagram of the associated task group in the method embodiment of the present invention;

Fig. 3 is the flowchart of the hierarchical scheduling of associated task groups in the method embodiment of the present invention;

Fig. 4 is a schematic structural diagram of an embodiment of the device of the present invention.

Detailed ways

The present invention will be further elaborated below in conjunction with the accompanying drawings and specific embodiments.

The present invention provides an embodiment of a cloud computing associated task scheduling method based on time constraints. FIG. 1 is an overall framework diagram of this embodiment, aiming at a set of associated task groups composed of n cloud computing associated tasks, each of which Tasks have their predecessors or successor tasks. Only when the predecessor task is completed, the successor task can start to be scheduled and executed. The user expects the completion time of the associated task group to be D. The main steps of the embodiment of the present invention are as follows:

(A) the associated task group enters the DAG graph building module, and the DAG graph building module builds the DAG graph according to the execution order of the tasks;

(B) The completed DAG graph enters the associated task layering module, and tasks are divided into task sets of different levels;

(C) Extract CPU resources, memory resources, network resources and storage resources in the resource pool to generate multiple virtual machines to form a virtual machine cluster;

(D) The task scheduling module assigns tasks at the same level to the virtual machines in the cluster according to a certain scheduling policy.

Wherein the concrete steps of step (A) are:

(A1) Define G={T,Λ,C} as a triplet describing the associated task group, where T is all task sets in the associated task graph T={t ₁ ,t ₂ ,...,t _n }, Λ is the set of execution order among tasks (t _i , t _j ), t _i is the direct predecessor task of t _j , t _j is the direct successor task of t _i , C={c ₁ ,c ₂ ,. ..,c _L } is the set of communication overhead between tasks. The greater the communication overhead between tasks, the longer the time for data transmission between tasks. When two tasks with a direct relationship are assigned to the same virtual machine for execution, the communication overhead is 0, and the time for transferring data from the predecessor task to the successor task can be effectively reduced.

(A2) Obtain the DAG graph of the associated task group according to the execution order of the tasks in the set Λ.

As shown in Figure 2, it intuitively shows the DAG graph with 10 associated tasks. This group of associated tasks is t ₁ -t ₁₀ . The association relationship between tasks is represented by arrows, and the numbers on the arrows represent the communication overhead between tasks. . Task t ₁ is the entry node of the associated task graph, task t ₁₀ is the exit node of the associated task, the direct predecessor task of tasks t ₂ , t ₃ , and t ₄ is t ₁ , and the successor tasks are t ₅ , t ₆ , and t _{7 respectively} , and so on, each task has its own predecessor or successor task.

The concrete steps of above-mentioned steps (B) are:

(B1) Divide the DAG graph into k layers according to the sequence from the entry task to the exit task.

(B2) Divide the tasks in the DAG diagram into the task sets of the corresponding layers according to their corresponding levels, as shown in Figure 2, the DAG diagram has t ₁ -t ₁₀ 10 associated tasks, according to from t ₁ to t ₁₀ By traversing the order of the task graph, the associated task graph can be divided into 5 layers: the first layer is t ₁ ; the second layer is t ₂ , t ₃ , t ₄ ; the third layer is t ₅ , t ₆ , t ₇ ; The fourth layer is t ₈ , t ₉ ; the fifth layer is t ₁₀ . For task sets at the same level, since there is no direct execution sequence correlation between tasks, tasks at the same level can be scheduled at the same time when the predecessor tasks are completed. For example, tasks t ₂ , t ₃ , and t ₄ can be scheduled simultaneously only when their predecessor task t ₁ is completed.

(B3) Take the completion time D of the associated task group expected by the user as the total time constraint of the associated task scheduling, according to The formula calculates the time constraints for each layer of task sets.

Among them, D _i is the constraint time of the i-th layer task set, m _i is the number of tasks in the i-th layer task set, and

The task scheduling process in the above-mentioned step (D) is described in detail below in conjunction with FIG. 3, including the following steps:

S401: Receive a layer of task set S _i from the associated task group that has divided the task set, and its time constraint is D _i , where i is the level of the currently received task set, and i=1 in the first scheduling;

S402: Assign the tasks in the task set S _i to different idle virtual machines in the virtual machine cluster for execution in sequence;

S403: Determine whether all the tasks in the task set S _i have been executed, if not, continue to wait for the tasks to be executed, otherwise go to S404;

S404: Determine whether there is a task set to be executed at the next level in the task group, if not, all the task groups are executed, otherwise go to S405;

S405: Calculate the total completion time T _i of all tasks in the task set S _i .

S406: Determine whether T _i < D _i is true, if it is true, take the next level of task set and go to step S402, otherwise take the next level of task set and go to step S407;

S407: Assign the tasks in the task set S _i to the same virtual machine as its direct predecessor tasks for execution, and go to step 403;

The present invention also provides an embodiment of a cloud computing-associated task scheduling device based on time constraints, as shown in FIG. ;

The task graph construction unit 101 is used to receive associated task groups, and build a DAG graph for all tasks according to the sequence of task execution;

The task hierarchy unit 102 is connected to the task graph construction unit 101, including a task set division unit 1021 and a time constraint calculation unit 1022. The task set division unit 1021 is used to divide tasks into different levels according to the DAG diagram constructed by the task construction unit 101. task set, the time constraint calculation unit 1022 calculates the time constraints of the associated task groups according to the proportion of the specific number of tasks in each layer;

The virtual machine management unit 103 is used to manage the virtual machine cluster, and can monitor the execution of tasks on the virtual machine;

The task scheduling unit 104 is connected to the task layering unit 102 and the virtual machine management unit 103, including a task set receiving unit 1041, a task set allocation unit 1042, a task set execution monitoring unit 1043 and a task set time calculation unit 1044;

The task set receiving unit 1041 sequentially receives a layer of task sets and the time constraints of this layer of task sets from the task layering unit 102; the task set allocation unit 1042 allocates the tasks in the task set to virtual machines according to the allocation rules, and the task set execution monitoring unit 1043 Monitor the execution of tasks, the task set time calculation unit 1044 calculates the completion time of all tasks in a task set and compares it with the constraint time, the task set allocation unit 1042 determines the allocation of the next layer of task sets according to the comparison result, if the actual completion time is less than the constraint time, the tasks in the next-level task set are assigned to different idle virtual machines in turn, otherwise, the tasks in the next-level task set are assigned to the virtual machines where their direct predecessor tasks are located.

The present invention has been described according to the preferred embodiments, it should be understood that the above embodiments do not limit the present invention in any form, and all technical solutions obtained in the form of equivalent replacement or equivalent transformation all fall within the protection scope of the present invention .

Claims (8)

1. the cloud computing associated task dispatching method based under time constraint condition, is characterized in that, comprises the following steps:

(1) all tasks in associated task group are set up to DAG figure according to the precedence of its execution;

(2) according to the DAG figure having built in step (1), all tasks being divided into different task-set by level, is every layer of task-set limiting time constraint;

(3) hardware resource is virtual, build cluster virtual machine;

(4) according to the time-constrain situation of every layer of task-set by task scheduling to suitable virtual machine in cluster virtual machine.

2. the cloud computing associated task dispatching method based under time constraint condition according to claim 1, is characterized in that, the concrete steps of described step (1) are:

(1.1) definition G={T, Λ, C} is the tlv triple of describing associated task group, and wherein, T is set of tasks all in associated task group, and Λ is the set of the execution order between task, and C is the set of communication overhead between task;

(1.2) show that according to the execution order between task in set Λ the DAG of associated task group schemes.

3. the cloud computing associated task dispatching method based under time constraint condition according to claim 1, is characterized in that, the concrete steps of described step (2) are:

(2.1) according to the order from entrance task to export task by described DAG figure layering;

(2.2) task in described DAG figure is divided in the set of tasks of equivalent layer according to its corresponding level;

(2.3) the associated task group deadline of user being expected, as total time-constrain of associated task scheduling, calculates the time-constrain of every layer of set of tasks according to formula.

4. the cloud computing associated task dispatching method based under time constraint condition according to claim 3, is characterized in that,

The formula that calculates the time-constrain of every layer of set of tasks in described step (2.3) is

Wherein, D is the confinement time that associated task group completes, D _ibe the confinement time of i layer set of tasks, n is the total task number in associated task group, m _ibe the number of tasks in i layer set of tasks, meet k is total number of plies of described DAG figure.

5. the cloud computing associated task dispatching method based under time constraint condition according to claim 1, is characterized in that, the concrete steps of described step (3) are:

(3.1) by physical resource, comprise cpu resource, Internet resources, memory source and storage resources, virtual and build many virtual machines;

(3.2) in virtual machine, dispose user task and carry out the required data of needed operating system, application program and tasks carrying;

(3.3) virtual machine building is added in cluster virtual machine to the wait task execution that is scheduled.

6. the cloud computing associated task dispatching method based under time constraint condition according to claim 1, is characterized in that, the flow process of the task scheduling in described step (4) is:

(4.1) receive one deck task-set according to the associated task group of having divided task-set in described step (2) by level, the task in task-set is dispensed to idle virtual machines different in cluster virtual machine, go to step (4.2);

(4.2) judge whether the task in received task-set is all finished, and continues wait task be finished if do not complete, otherwise go to step (4.3);

(4.3) total deadline of the calculating all tasks in task-set that receive, judge whether to be less than the confinement time of this layer of task, if actual total deadline is less than confinement time, goes to step (4.1), otherwise go to step (4.4);

(4.4) task in lower one deck task-set is dispensed in the virtual machine identical with its predecessor task, goes to step (4.2).

7. the cloud computing associated task dispatching device based under time constraint condition, it is characterized in that, comprise task image construction unit (101), task layering unit (102), Virtual Machine Manager unit (103) and task scheduling unit (104);

Described task image construction unit (101) is for receiving associated task group, and all tasks are set up to DAG figure according to the precedence of tasks carrying;

Described task layering unit (102) is connected with task image construction unit (101), comprise task-set division unit (1021) and time-constrain computing unit (1022), described task-set division unit (1021) is divided into different task-set by task according to level for the DAG figure having built according to described task construction unit (101), and described time-constrain computing unit (1022) is assigned to the time-constrain of associated task group the time-constrain of concrete each layer;

The implementation status of task on virtual machine, for managing virtual machines cluster, can be monitored in described Virtual Machine Manager unit (103);

Described task scheduling unit (104) is connected with Virtual Machine Manager unit (103) with task layering unit (102) respectively, for the task-set of having divided by layer of task layering unit (102) is dispensed to the suitable virtual machine of Virtual Machine Manager unit (103) management according to time-constrain situation.

8. the cloud computing associated task dispatching device based under time constraint condition according to claim 7, is characterized in that,

Described task scheduling unit (104) comprises task-set receiving element (1041), task-set allocation units (1042), task-set execution monitoring unit (1043) and task-set time calculating unit (1044);

Described task-set receiving element (1041) receives the time-constrain of pending task-set and task-set from described task layering unit (102);

Described task-set allocation units (1042) are dispensed to the task in task-set in virtual machine and carry out, described task-set is carried out monitoring unit (1043) monitor task implementation status, described task-set time calculating unit (1044) calculation task concentrates all tasks complete the time of expense and compare with the confinement time of task-set, and described task-set allocation units (1042) are determined the distribution condition of lower one deck task-set according to comparative result.