CN110489229B

CN110489229B - Multi-target task scheduling method and system

Info

Publication number: CN110489229B
Application number: CN201910645478.2A
Authority: CN
Inventors: 周舟; 李方敏; 刘萍; 张韬; 杨志邦; 姚文静
Original assignee: Changsha University
Current assignee: Changsha University
Priority date: 2019-07-17
Filing date: 2019-07-17
Publication date: 2020-06-30
Anticipated expiration: 2039-07-17
Also published as: CN110489229A

Abstract

The invention discloses a multi-target task scheduling method and a system, wherein fuzzy clustering processing is respectively carried out on tasks and virtual machines by utilizing a fuzzy clustering algorithm, an obtained task clustering set is matched with a virtual machine clustering set to obtain a matching set, a differential evolution algorithm is adopted to schedule the tasks in the matching set to the virtual machines in the matching set to obtain an initial scheduling result, the virtual machines are divided into a heavy load set, a balanced load set and a light load set according to the load value of the virtual machines in the initial scheduling result, the task set needing to be rescheduled is selected from the heavy load set to obtain a rescheduled task set, a Q value algorithm is adopted to schedule the tasks in the rescheduled task set to the virtual machines in the light load set, the technical problem of poor scheduling strategy obtained by adopting the traditional task scheduling algorithm is solved, local resource tasks are redistributed by utilizing a Q value method, the load balance of global resources is realized, and the resource utilization rate is improved.

Description

Multi-target task scheduling method and system

Technical Field

The invention relates to the technical field of cloud computing, in particular to a multi-target task scheduling method and system.

Background

Cloud computing is a market-oriented emerging technology based on network development, and becomes a hot topic of academic research in recent years. With the annual increase of the market share and the expansion of the resource scale, the huge user quantity and task quantity follow up, so that the difficulty is increased for reasonably and efficiently scheduling the tasks in the cloud environment. And the excellent scheduling strategy is designed to play an indispensable role in improving the running performance of the cloud platform, shortening the task response time and improving the service satisfaction of the user.

The scale and the quantity of tasks submitted to the cloud platform by the users are huge, the tasks submitted by a plurality of users do not have constraint relation, the tasks exist in a parallel mode, and how to reasonably distribute the tasks with huge quantity to massive resource nodes is a difficult and complex process. Cloud task scheduling belongs to a multi-objective combination optimization problem and is affected by multiple factors, a traditional task scheduling algorithm is only optimized for a single objective in a scheduling process, and the algorithm has certain defects when processing a large number of tasks, so that the obtained result scheme of the scheduling strategy is not ideal.

Cloud computing resources have great differences, and they have differences not only in geographic positions but also in performance. The cloud computing system integrates heterogeneous resources into a whole through virtualization, a resource pool capable of realizing sharing is formed, and when a user has a task demand, the user can allocate corresponding virtual resources to the user according to the demand of the user. The assignment of a large number of task sets in a large resource set is subject to blindness and limitations. In addition, the tasks submitted by the user also have various preferences, such as: the computational task is biased to high-complexity scientific computation; storage-type tasks require a large amount of storage space to store large data; the large amount of bandwidth type task communication data requires high bandwidth access; data intensive tasks require large amounts of computational and storage resources for big data statistical analysis mining and the like. Therefore, resources having different types of advantages should be preferentially allocated to tasks having corresponding preferences.

In a cloud computing environment, the goals of users and cloud service providers vary. Users pay more attention to task completion time, and cloud service providers often pay more attention to resource utilization rate, task execution cost, energy consumption and other aspects. Therefore, it is a considerable problem to schedule tasks to appropriate resources to execute and simultaneously meet the requirements of various parties in the cloud environment.

Disclosure of Invention

The multi-target task scheduling method and system provided by the invention solve the technical problem of poor scheduling strategy obtained by adopting the traditional task scheduling algorithm.

In order to solve the technical problem, the multi-target task scheduling method provided by the invention comprises the following steps:

respectively carrying out fuzzy clustering processing on the tasks and the virtual machines by using a fuzzy clustering algorithm to obtain a task clustering set and a virtual machine clustering set, and matching the task clustering set with the virtual machine clustering set to obtain a matching set;

scheduling the tasks in the matching set to the virtual machines in the matching set by adopting a differential evolution algorithm to obtain an initial scheduling result;

dividing the virtual machines into a heavy load set, a balanced load set and a light load set according to the load values of the virtual machines in the initial scheduling result;

selecting a task set needing rescheduling from the heavy load set to obtain a rescheduling task set;

and scheduling the tasks in the rescheduling task set to the virtual machines in the light load set by adopting a Q value algorithm.

Further, matching the task cluster set with the virtual machine cluster set to obtain a matching set includes:

according to the computing power, the bandwidth and the storage capacity of the virtual machines in the virtual machine cluster set, performing weighted computation to obtain the resource performance of the virtual machines;

acquiring the comprehensive resource performance of the virtual machine cluster set according to the average value of the resource performance of all the virtual machines in the virtual machine cluster set;

according to the length and the size of the tasks in the task clustering set and the size of output data, performing weighted calculation to obtain the resource requirements of the tasks;

acquiring the comprehensive resource requirements of the task clustering set according to the average value of the resource requirements of all tasks in the task clustering set;

and matching the task cluster set with the virtual machine cluster set according to the distance between the comprehensive resource performance of the virtual machine cluster set and the comprehensive resource requirement of the task cluster set to obtain a matching set.

Further, scheduling the tasks in the matching set to the virtual machines in the matching set by adopting a differential evolution algorithm, and obtaining an initial scheduling result comprises:

encoding individuals of a population in the differential evolution algorithm by adopting a one-dimensional group according to the number of tasks, the number of virtual machines and the number of tasks in the matching set;

calculating a variation factor and a cross factor according to the current evolution algebra;

performing variation operation on population individuals according to the variation factors and combining various variation strategies to obtain variant individuals;

selecting the components of the parent individuals or the variant individuals according to the cross factors to obtain cross individuals;

reversely selecting the components of the parent individuals or the variant individuals according to the cross factors to obtain anti-cross individuals;

determining a moderate function according to task completion time, task execution cost and virtual machine load balance;

and obtaining the appropriate values of the target individual, the variant individual, the cross individual and the anti-cross individual according to the appropriate function, and obtaining the next generation individual according to the appropriate values, thereby decoding the next generation individual to obtain an initial scheduling result.

Further, the concrete formula for calculating the variation factor and the cross factor according to the current evolution algebra is as follows:

wherein F (T) represents a mutation factor, CR (T) represents a cross factor, w (T) represents a weight factor, T represents an evolution algebra, T represents a population maximum evolution algebra, F (T) represents a mutation factor, CR (T) represents a cross factor, w (T) represents a weight factor, T represents a population maximum evolution algebra_maxDenotes the maximum value of the variation factor, F_minRepresents the minimum value of the variation factor, CR_minDenotes the minimum value of the crossing factor, CR_maxRepresenting the cross factor maximum.

Further, according to the variation factor, combining with various variation strategies, performing variation operation on the population individuals, and obtaining a specific formula of the variation individuals as follows:

wherein λ represents a weight factor, u represents an influence factor, v_i(T) represents the variant individual corresponding to the population current evolution algebra, T represents the population maximum evolution algebra, x_iRepresenting the current individual, x_r1(t)、x_r2(t)、x_r3(t)、x_r4(t) represents four differences from x corresponding to the current evolutionary algebra of the randomly selected population_iIndividual of (1), x_best(t) represents the best individual in the contemporary population, and F represents the variation factor.

Further, obtaining the appropriate values of the target individual, the variant individual, the crossover individual and the anti-crossover individual according to the appropriate function, and obtaining the calculation formula of the next generation individual according to the appropriate values specifically comprises:

where f denotes a moderating function, and f ═ min [ a ln (totalTime) + b ln (total Cost) + c ln (load)]，x_i ^t+1Representing the next generation of target individuals, u_i ^t+1Represents the next generation of crossover individuals, f (u)_i ^t+1) Representing the fitness of the next generation of crossover individuals, v_i ^t+1F (v) representing the next generation of variant individuals_i ^t+1) Represents the moderate value of the next generation variant individual, h_i ^t+1Representing the next generation of anti-crossover individuals, f (h)_i ^t+1) Representing the fitness of the next generation of anti-crossover individuals, f (x)_i ^t) Representing the fitness value, x, of the current generation of target individuals_bestRepresents individuals with the best fitness value in the contemporary population, x_avgThe method comprises the steps of representing the median of all individuals in a current generation population, totalTime representing task completion time, totaLcost representing task operation cost overhead, load representing load balance degree, a, b and c being weight factors, and meeting the condition that a + b + c is 1.

Further, the formula for calculating the Q value in the Q value algorithm is specifically as follows:

wherein Q is_iDenotes the Q value after assignment of the task to the jth virtual machine, vm_pjRepresenting the processing power of the jth virtual machine in the lightly loaded set, vmTime_jRepresents the execution time, taskTime, of the task on the jth virtual machine in the light load set_ijAnd the task execution time for migrating the ith task in the rescheduled task set to the jth virtual machine in the light load set is shown.

Further, the maximum value of the variation factor F_max1, minimum value of the variation factor F_min0.4, minimum value of crossing factor CR_min0.6, maximum value of cross factor CR_max＝0.9。

The invention provides a multi-target task scheduling system, which comprises:

the multi-target task scheduling method comprises the following steps of a memory, a processor and a computer program which are stored on the memory and can run on the processor, wherein the steps of the multi-target task scheduling method are realized when the processor executes the computer program.

Compared with the prior art, the invention has the advantages that:

the invention provides a multi-target task scheduling method and system, which respectively carry out fuzzy clustering processing on tasks and virtual machines by utilizing a fuzzy clustering algorithm to obtain a task clustering set and a virtual machine clustering set, match the task clustering set with the virtual machine clustering set to obtain a matching set, schedule the tasks in the matching set to the virtual machines in the matching set by adopting a differential evolution algorithm to obtain an initial scheduling result, divide the virtual machines into a heavy load set, a balanced load set and a light load set according to the load value of the virtual machines in the initial scheduling result, select a task set needing to be rescheduled from the heavy load set to obtain a rescheduled task set, and schedule the tasks in the rescheduled task set to the virtual machines in the light load set by adopting a Q value algorithm, thereby solving the technical problem of poor scheduling strategy obtained by adopting the traditional task scheduling algorithm, by utilizing a Q value method to redistribute local resource tasks, the load balance of global resources is realized, and the resource utilization rate is improved.

Drawings

FIG. 1 is a flowchart of a multi-target task scheduling method according to a first embodiment of the present invention;

FIG. 2 is a flowchart of a multi-target task scheduling method according to a second embodiment of the present invention;

FIG. 3 is a clustering result obtained after fuzzy clustering is performed on tasks and resources for a hypothetical cloud task and virtual machine according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of task scheduling obtained for a hypothetical cloud task and virtual machine according to an embodiment of the present invention;

FIG. 5 is a task scheduling scheme obtained by decoding hypothetical cloud tasks and virtual machines according to an embodiment of the present invention;

FIG. 6 is a diagram of a cross operation process according to a second embodiment of the present invention;

FIG. 7 is a diagram of a reverse interleaving operation of the second embodiment of the present invention;

FIG. 8 is a working process of the t iteration of the population in the IADE algorithm according to the second embodiment of the present invention;

FIG. 9 is a flowchart of a multi-target task scheduling method according to a third embodiment of the present invention;

FIG. 10 is a graph comparing the execution time results of the clustering-free DE algorithm and the clustered DE algorithm in the third embodiment of the present invention;

FIG. 11 is a comparison graph of task completion times for different numbers of tasks according to a third embodiment of the present invention;

FIG. 12 is a comparison graph of the number of tasks of the virtual machine according to the third embodiment of the present invention;

FIG. 13 is a comparison chart of task execution costs according to a third embodiment of the present invention;

FIG. 14 is a comparison graph of load balancing values according to a third embodiment of the present invention;

FIG. 15 is a block diagram of a multi-objective task scheduling system, in accordance with an embodiment of the present invention.

Reference numerals:

10. a memory; 20. a processor.

Detailed Description

In order to facilitate an understanding of the invention, the invention will be described more fully and in detail below with reference to the accompanying drawings and preferred embodiments, but the scope of the invention is not limited to the specific embodiments below.

The embodiments of the invention will be described in detail below with reference to the drawings, but the invention can be implemented in many different ways as defined and covered by the claims.

Example one

Referring to fig. 1, a multi-target task scheduling method provided in an embodiment of the present invention includes:

step S101, fuzzy clustering processing is respectively carried out on tasks and virtual machines by utilizing a fuzzy clustering algorithm to obtain a task clustering set and a virtual machine clustering set, and the task clustering set is matched with the virtual machine clustering set to obtain a matching set;

step S102, scheduling tasks in the matching set to virtual machines in the matching set by adopting a differential evolution algorithm to obtain an initial scheduling result;

step S103, dividing the virtual machines into a heavy load set, a balanced load set and a light load set according to the load values of the virtual machines in the initial scheduling result;

step S104, selecting a task set needing rescheduling from the heavy load set to obtain a rescheduling task set;

and step S105, scheduling the tasks in the rescheduling task set to the virtual machines in the light load set by adopting a Q value algorithm.

The multi-target task scheduling method provided by the embodiment of the invention obtains a task cluster set and a virtual machine cluster set by respectively carrying out fuzzy clustering processing on tasks and virtual machines by using a fuzzy clustering algorithm, matches the task cluster set with the virtual machine cluster set to obtain a matching set, schedules the tasks in the matching set to the virtual machines in the matching set by using a differential evolution algorithm to obtain an initial scheduling result, divides the virtual machines into a heavy load set, a balanced load set and a light load set according to the load value of the virtual machines in the initial scheduling result, selects the task set needing to be rescheduled from the heavy load set to obtain a rescheduled task set, and schedules the tasks in the rescheduled task set to the virtual machines in the light load set by using a Q value algorithm, thereby solving the technical problem of poor scheduling strategy obtained by using the traditional task scheduling algorithm, by utilizing a Q value method to redistribute local resource tasks, the load balance of global resources is realized, and the resource utilization rate is improved.

Specifically, since the tasks submitted by the users in this embodiment have multiple types and the scheduling problem studied in the embodiment of the present invention is directed to an independent task, in order to establish an effective scheduling model, the following assumptions are made:

(1) tasks uploaded by different users are mutually independent and have no constraint relation;

(2) the total number of tasks submitted by the user is far larger than the number of resources in the cloud environment;

(3) the virtual machine executes tasks in a non-preemptive manner according to a first-come-first-serve principle;

(4) independent tasks are not resolvable and can only be assigned to one virtual machine.

According to the hypothesis, the data dependency relationship does not exist among the subtasks in the independent task set, the task submitted to the cloud platform by the user is assumed to have n subtasks, the subtasks are numbered in sequence, the independent task set is represented by T,

the task model is defined as:

T＝{t₁,t₂,t₃,...,t_n} (1)

t_i＝{t_Id,t_Length,t_FileSize,t_OutputSize} (2)

wherein, t_IdA number representing a task to uniquely identify each task; t is t_LengthIndicating the length of the task in units of MI (Million instruments); and t is_FileSizeRepresented is the size of the task, in units of MB; t is t_OutputSizeRepresenting the required output data size.

Cloud meterHeterogeneous computing resources of the computing data center are abstractly combined into a cluster, and m virtual machines with different performances are assumed to exist in a resource pool of the cloud data center and are represented by a set VM (virtual machine model), VM_jRepresenting the jth virtual machine, the resource model may be represented as:

VM＝{vm₁,vm₂,vm₃,...,vm_m} (3)

vm_j＝{vm_Id,vm_Mips,vm_PENumber,vm_Stor,vm_BW} (4)

wherein, the virtual machine performance information set includes: vm_IdThe id representing the virtual machine is the unique identifier of each virtual machine; vm_MipsThe number of instruction executions (million instructions per second) representing the virtual machine; vm_PENumberIndicates the number of CPUs vm_StorRepresenting virtual machine storage capacity; vm_BWRepresenting the virtual machine bandwidth size.

vm_jThe calculation power of (c) can be found by the following formula:

vm_jcompute＝vm_PENumber*vm_Mips(5)

after the distribution of the task set in the virtual machine set is completed, the execution time of each task on the distributed virtual machine can be calculated, and the execution time is expressed by a matrix ETC as follows:

wherein m and n are respectively the number of virtual machines and tasks, etc_j,iAs task t_iIn virtual machine vm_jThe execution time of.

In the multi-objective optimization problem, the influence factors are restricted with each other, so that the optimization targets have a restriction relationship, wherein the increase of one optimization target value may cause the decrease of other optimization target values which are manufactured by the optimization target, and the simultaneous optimization of all the targets is often impossible. Therefore, the solution of the multi-objective optimization problem requires a comprehensive consideration of all objectives so that they are as optimal as possible in the conditions that are constrained to each other.

In a cloud computing environment, the completion time of a task is a key goal of common concern for users as well as cloud service providers. As a consumer, the task cost is the focus of attention of the user, and the cost of the user can be indirectly reduced by reducing the completion time of the task. Meanwhile, cloud computing is commercial, and for providers, resource utilization rate can be improved by achieving load balance of the system, waiting time of users is reduced, and satisfaction of the users is improved. Therefore, the benefits of the user and the service provider are comprehensively considered, the task scheduling optimization target selected by the embodiment includes the user task completion time, the task running cost and the load balance of the virtual machine, and an optimal scheduling scheme is sought to be achieved simultaneously, so that the service quality requirement of the user is guaranteed, the resource utilization rate is improved, and the waste of cloud resources is avoided.

(1) Task completion time

The execution time of a single virtual machine j, which is the time it takes for all assigned tasks thereon to complete, can be expressed as:

the virtual machines in the cloud environment are executed in a parallel manner, so that the total time consumed by task execution is the time consumed by the virtual machine which spends the longest time in all the virtual machines performing the execution task, and the task completion time can be expressed as:

totalTime＝max(vmTime_j) (8)

(2) task running cost overhead

In the present embodiment, the task operation cost overhead is defined as the sum of the task execution cost and the storage cost.

The cost of executing the task is the sum of the product of the execution time and the unit execution price of all the virtual machine tasks, and can be expressed as:

in the formula, vmTime_jRepresenting the execution time, vm, of the task assigned by virtual machine j_jCPUpriceThe execution price per unit time of virtual machine j is indicated.

Task t_iIn virtual machine vm_jThe storage fee above may be expressed as:

StorCost_i,j＝t_iFileSize*vm_jStorprice(10)

in the formula, t_iFileSizeRepresents the size of the memory occupied by task i, vm_jStorpriceRepresenting virtual machines vm_jThe unit size of (a) stores the price.

The total storage cost of the tasks is the sum of the storage costs of all the virtual machine tasks:

after the formula (9) and the formula (11) are combined, the task runtime cost totalCost may be recorded as:

total Cost＝CPU Cost+Stor Cost (12)

(3) virtual machine load

The load of virtual machine j in this embodiment is represented by the total time to perform the assigned task, virtual machine vm_jThe load at a certain moment can be formulated as:

the total load and the average load of all virtual machines in the system can be expressed as:

the load balance degree in the system is expressed by standard deviation and can be defined as:

the load balance degree load is used for quantifying the load condition of the virtual machine cluster in the cloud system, the load is 0 when the loads of all the computing nodes are the same at a certain moment, and the virtual machine cluster is in an ideal load balance state at the moment, and the smaller the load is, the more balanced the load is.

The task scheduling strategy based on multi-objective optimization in the embodiment has the following targets:

example two

Referring to fig. 2, a multi-target task scheduling method provided by the second embodiment of the present invention includes:

step S201, fuzzy clustering processing is respectively carried out on the tasks and the virtual machines by utilizing a fuzzy clustering algorithm to obtain a task clustering set and a virtual machine clustering set, and the task clustering set is matched with the virtual machine clustering set to obtain a matching set.

With the rapid development of cloud computing, the scale of resource clusters is continuously enlarged. Task scheduling in a current cloud environment usually aims at all resources, a relationship between tasks and resources is not considered, when resources are abundant and the quantity of tasks is large, time overhead for allocating the tasks to appropriate resource nodes is increased, which directly has a great influence on the final completion time of the tasks and also causes reduction of task execution efficiency. Considering that the blindness of task selection resources has a certain influence on the task execution condition, the embodiment sets a preprocessing stage and performs cluster partitioning solution on the tasks and the resources at the same time when scheduling the tasks.

When user tasks and cloud resources are clustered, nodes with similar resource performance attributes are classified into one class, task resource requirements are classified into one class, after division, each small task set corresponds to one small resource node set, and the scales of the tasks and the resources are reduced. When the tasks are scheduled, the corresponding computing node sets can be accurately selected by the tasks, the time for searching the resources is shortened, the task waiting time is further shortened, the distribution process is efficient, the nodes in the high-performance resource sets can process complex task demands, and the nodes in the low-performance resource sets can process simple tasks.

The resources in the cloud computing environment have heterogeneity and are obvious in ambiguity, and strict division is not performed between the resources, so that uncertainty of the resources can be reflected more accurately by dividing the resources by using a fuzzy clustering method. The embodiment uses a fuzzy clustering method to complete the clustering of tasks and resources.

Specifically, in this embodiment, the fuzzy clustering algorithm is used to perform fuzzy clustering processing on the task and the virtual machine respectively to obtain a task cluster set and a virtual machine cluster set, and the task cluster set is matched with the virtual machine cluster set to obtain a matching set, where the obtaining of the matching set includes:

step S2011, according to the computing power, bandwidth, and storage capacity of the virtual machines in the virtual machine cluster set, performing weighted computation to obtain the resource performance of the virtual machines.

Specifically, the virtual machine vm in the virtual machine cluster set of this embodiment_jResource performance of by vm_jThe following formula is obtained by combining the capabilities of the following components:

in the formula, the parameters a, b and c are used to represent vm to the virtual machine_jCalculated performance, bandwidth and stored empirical coefficients.

Step S2012, obtaining the comprehensive resource performance of the cluster set of the virtual machines according to the average value of the resource performance of all the virtual machines in the cluster set of the virtual machines.

Specifically, the comprehensive resource performance of the cluster set of virtual machines in this embodiment is an average value of the resource performances of all virtual machines in the set, and the formula is as follows:

in the formula, the parameter n represents the number of virtual machines in the resource set.

And S2013, performing weighted calculation to obtain the resource requirements of the tasks according to the length and the size of the tasks in the task clustering set and the size of output data.

Specifically, the resource requirement calculation formula of task i in the task cluster set of this embodiment is as follows:

in the formula, the parameters a, b and c represent the task t respectively_iThe amount of demand for computation, bandwidth and storage capacity.

Step S2014, obtaining the comprehensive resource requirement of the task clustering set according to the average value of the resource requirements of all the tasks in the task clustering set, wherein the specific formula is as follows:

step S2015, matching the task cluster set and the virtual machine cluster set according to a distance between the comprehensive resource performance of the virtual machine cluster set and the comprehensive resource demand of the task cluster set, and obtaining a matching set.

Specifically, the embodiment aggregates the aggregate resource requirement t of the set according to the task_REIntegrated resource performance r with a clustered set of virtual machines_GPThe distance d between the task sets is a proper resource set selected for the task set, and the smaller the distance is, the more the resource of the set can meet the task requirement in the task set.

d＝|r_GP-t_RE| (22)

In the preprocessing stage, tasks and resources are classified, as shown in fig. 3, and the scales of the tasks to be matched and the virtual machine resources are reduced, so that the matching time in the scheduling process can be greatly reduced, the task scheduling efficiency and the user satisfaction are improved, and a multi-objective task scheduling strategy of an improved differential evolution algorithm is used for scheduling in a subsequent matching set.

And S202, encoding individuals of the population in the differential evolution algorithm by adopting a one-dimensional group according to the number of tasks, the number of virtual machines and the number of tasks in the matching set.

When using a differential evolution algorithm for task scheduling, individuals of a population need to be encoded. The encoding method of the embodiment is as follows: the number of tasks is M and is numbered as [0, M-1 ]](ii) a The number of the virtual machines is N and the number is [0, N-1 ]]. The group individual set is represented by a plurality of one-dimensional arrays, the length of the one-dimensional array is the number of tasks, the index of the array is the number of the tasks, the value corresponding to the array index is the number of the virtual machine, and the range of the value is [0, N-1 ]]And indicating that the task with the number of the array subscript is dispatched to the virtual machine on the value corresponding to the array subscript to be executed. Each array represents a task scheduling scheme, and the population size NPs constitute NP scheduling schemes. When the algorithm randomly initializes a seed group, the number result of the generated virtual machine is a decimal number, so that the obtained optimal scheduling scheme is not in an integer coding form, and further coding processing is needed. Suppose that the user submits cloud tasks of { T1, T2, T3, T4, T5, T6}, and the system has virtual machines of { VM1, VM2, VM3 }. When the optimal solution obtained by the algorithm operation is coded as X (2.5122,0.3196,1.8350,0.8705,2.2845 and 0.2445), the calculation is carried out on each dimension component of the individual to obtain the integer coding form

The integer-coded individuals are shown in fig. 4. The decoding results in the final task scheduling scheme as shown in fig. 5.

Step S203, calculating the variation factor and the cross factor according to the current evolution algebra.

The mutation operation is to mutate individual population to increase diversity of the population. The target individual is disturbed to generate a variant individual, so that the population can be prevented from falling into local optimum. In the variation operation step, in the t-th iteration, two different individuals are randomly selected from the population, subtracted to generate a random difference vector, weighted, and then summed with a third randomly selected population to generate a variation individual, and the variation individual is generated by the summation operation of the random difference vector and the third randomly selected population individualFor a certain individual vector in the parent population, an individual v obtained through mutation_iThe calculation method is as follows:

v_i(t)＝x_r1(t)+F*(x_r2(t)-x_r3(t)) (23)

in the formula, t represents the algebra of the current population evolution, the sequence numbers r1, r2 and r3 are three indexed individuals with different sequence numbers from the sequence number of the current individual i randomly selected from the population individual set, and the individual x is ensured at the same time_r1、x_r2、x_r3Each different from the others, by random individuals x_r1As basis vectors, by x_r2And x_r3And after weighting the random vector difference, carrying out disturbance to generate a variation vector. F is a variation factor with a value of [0,2]The method is used for controlling the influence degree of the random difference vector, and is the key for ensuring the individual mutation. When the vector difference of the two individuals is smaller, the similarity of the individuals is larger, and the whole population gradually approaches to the optimal solution. The magnitude of the value of the variation factor F has an effect on the global optimization capability of the algorithm. When the F value is smaller, the algorithm is more biased to local search, and the area range of possible solution search is smaller; when the F value is large, the algorithm cannot generate search stagnation at a local extreme point, but the convergence speed is slowed down due to the large F value.

At present, the DE algorithm has many different modes of mutation strategies, and is generally expressed in the form of "DE/a/b", where a denotes how to select individuals participating in mutation operations from population individuals, and generally takes the values of rand, best, and current. Wherein rand represents that the individuals participating in the mutation operation are random individuals, best represents that the mutation operation is mainly based on the individuals corresponding to the current optimal solution, and current represents that the mutation operation is performed by taking the current target individuals of the population as basis vectors; b represents the number of terms of the difference term used in the mutation operation, and is generally an integer. The mode in the standard DE algorithm is denoted by "DE/rand/1", and besides the DE algorithm, there are five variation strategies shown in Table 1.

TABLE 1 DE Algorithm mutation strategy

X involved in mutation strategies_r1、x_r2、x_r3、x_r4、x_r5Representing random individual vectors, x, within the population that are different from the current individual and each other_bestAnd F is a variation factor.

For different variation strategies, the performance difference of the differential evolution algorithm is large, the basis vectors of the DE/rand/1 and DE/rand/2 variation strategies are random individuals, the search range is wide, and the optimal value search speed is slow; DE/best/1 and DE/best/2 regard optimum individual in the contemporary population as the vector of basis, search range limit in optimum individual nearby, apt to appear premature convergence phenomenon; the convergence and search capabilities of the DE/current-to-best/2 and DE/rand-to-best/2 mutation strategies are balanced.

The variation factor F of the differential evolution algorithm plays a role in adjusting the search scale and the population diversity of the algorithm, the algorithm is very sensitive to the setting of the variation factor F, and usually F is a real number which is valued in [0,2] and is used for controlling the amplitude of a differential term. If F is improperly set, a new variant individual cannot be generated in the variant stage, the similarity of the individuals increases and gradually converges, and meanwhile, F also affects the convergence rate of the algorithm: when the F is set to be larger, the difference degree between individuals is increased, the algorithm searching range is also increased at the moment, the algorithm searching range is close to a global random searching mode, the algorithm efficiency is too low due to slow convergence, and the quality of the obtained solution is not high; when the F is set to be smaller, the searching scale of the algorithm is reduced at the moment, the local searching is enhanced, and the algorithm falls into a local optimal solution although the convergence speed is increased. Therefore, reasonably setting the variation factor F has a great influence on the quality and convergence rate of the optimal solution of the algorithm. In the standard differential evolution algorithm, F is set as a fixed constant and remains fixed and unchanged in the whole iterative solution process, which leads the algorithm to be unable to meet the size requirements of parameters in each period in the evolution process, so that the algorithm is easily trapped in a local optimum value and unable to obtain global optimum, and the convergence rate is relatively slow. To solve these problems with the DE algorithm, the present embodiment sets F to be adaptive, automatically setting the size of the value as the evolution progresses. When the algorithm is iterated at the beginning, F is set to be a large value, so that the algorithm can have a large search space to ensure that the diversity of the population can avoid the escape of the global optimal value. In the later stage, along with the continuous increase of evolution algebra, the value of F is continuously reduced, and the algorithm can perform local detailed search in a small range with feasible solutions, so that the optimal solution is prevented from being damaged, and the searching precision is improved. The improved formula of F is:

F(t)＝F_max-(F_max-F_min)*w(t) (25)

the magnitude of the F value is set according to the current evolution algebra, the value of the weight factor w (t) is increased along with the increase of the evolution algebra t, and the magnitude of the newly assigned F value is decreased progressively, so that the local search is enhanced on the basis of the better solution obtained by the algorithm in the later stage of evolution, and the algorithm can be rapidly converged to the optimal solution. The effective interval of F is [0.4, 1%]Therefore, the maximum value F of the variation factor in this embodiment_max1, minimum value of the variation factor F_min＝0.4。

Crossover manipulation follows mutation manipulation, which is further manipulated in order to increase the diversity of population individuals. The cross operation determines the variant individuals V through a preset strategy_iThe dimension components are used for replacing the dimension components of the corresponding positions of the target individuals, and the CR determines the value source and the updating proportion of each dimension component of the crossed individuals, so that the diversity of the population in the algorithm is greatly influenced.

In the formula, the cross probability CR is taken in an interval [0,1], the size of CR determines the probability of cross components, the larger CR is, the larger the information amount is exchanged, and more dimension components of cross individuals come from variant individuals; the smaller the CR, the more the cross-member individual components will come from the parent individuals, so that the diversity of the population is maintained. D represents the dimension of the individual, rand (D) generates a random number in the dimension range of the individual, and j is used for ensuring that the crossover individual performs crossover operation at least once in the crossover step, so as to ensure that the crossover individual and the parent individual are not completely the same to avoid that the population cannot approach the optimal value before the evolution of the population is stopped. The specific interleaving procedure is shown in fig. 6. In fig. 6, each component of the crossover individual is determined to be a value according to the crossover factor CR, and is determined to be a parent target individual vector or a variant individual vector.

The diversity of population individuals can be increased by the cross operation of the DE algorithm, and a parameter cross factor CR in the cross strategy determines the magnitude degree of dimension components exchanged between children and parent and cross individuals. The larger CR indicates that the contribution rate of the variation variable is higher, the larger the disturbance is, the larger the proportion of each dimension component belonging to the variation vector in each dimension component of the cross vector is, so that the diversity of the population is reduced, and the algorithm is favorable for carrying out local search so as to accelerate the rate of convergence; the smaller the CR value is, the more favorable the preservation of population diversity and global search capability are. In the standard differential evolution algorithm, the CR as a fixed parameter cannot meet the preference of CR size setting at each stage in the evolution process. Therefore, in the evolution process, the value of the crossover factor CR needs to be changed continuously according to the operating conditions, and the improved algorithm specifically comprises:

CR(t)＝CR_min+(CR_max-CR_min)*w(t) (27)

the self-adaptive improvement strategy of the cross factor CR is the same as the variation factor F, the new value of the CR is also set according to the current evolution algebra t, the value of the weight factor w (t) is increased along with the increase of the evolution algebra t, and the newly assigned value of the CR is also increased, so that the convergence speed of the fitness value of the algorithm at the later stage of evolution can be accelerated, and the convergence precision is ensured. The minimum value of the crossover factor in this embodiment is CR_min0.6, maximum value of cross factor CR_max＝0.9。

And step S204, carrying out variation operation on the population individuals according to the variation factors and combining various variation strategies to obtain the variation individuals.

In the DE algorithm, a single mutation mode is often difficult to meet the requirements of population evolution, so that the respective characteristics can be integrated by adopting a mode of combining multiple strategies. The DE/rand/1 mode can ensure the diversity of the population, has stronger global search capability, is not easy to fall into the local optimum, but has slower convergence speed; while the DE/best/1 mode converges faster. The improved mutation strategy of the embodiment combines the two classical mutation modes, and can give consideration to both the search capability and the convergence at the same time, and the improved modes of the embodiment are as follows:

v_i(t)＝u*x_r1(t)+(1-u)*x_best(t)+F*(x_r2(t)-x_r3(t)) (29)

wherein T represents evolution algebra, T population maximum evolution algebra, x_iRepresenting the current individual, x_r1、x_r2、x_r3Representing three randomly selected ones different from x_iIndividual of (1), x_bestRepresenting the best individual in the contemporary population.

The size of the variable u changes along with the iteration times t, the image of u in the interval [0,1] is parabolic, and the early-stage numerical value increases slowly, so that the value of u also decreases slowly. Therefore, in the previous population evolution, u → 1, the variation mode can be inclined to DE/rand/1 to the maximum extent, and at the moment, the algorithm has stronger global search performance so that the population individual diversity can be ensured. In the later stage u → 0, the mutation strategy tends to be DE/best/1, and the performance of the algorithm can reach better balance in the search capability, stability and optimizing speed, so that the low convergence speed of the algorithm in the early stage is avoided. The improved algorithm can self-adaptively select different variation strategies at different stages in the population evolution process, and the search capability and the convergence speed are balanced.

Meanwhile, in order to better control the evolution direction of the later population, the embodiment adds an optimal difference value on the original basis, and adds weight factors to the random difference and the optimal difference to control the respective contribution rates. The improved mutation strategy formula is as follows:

Specifically, λ in this embodiment is a weight factor, which is used to control the random difference and the optimal difference to perturb the magnitude of the variation individual, where a value range of λ is an interval [0,1], and the magnitude is gradually increased with the population evolution algebra, so that the random difference has a larger weight in an early stage of evolution, and the contribution degree in the variation vector is maximum, and the algorithm can enhance the global search strength. In the later evolution stage, population individuals can be converged, so that the potential range of the optimal solution is reduced, and in order to accelerate the speed of searching the optimal solution, the diversity of the population individuals needs to be reduced, the proportion of the optimal difference is increased in a corresponding formula, so that the algorithm can perform oscillation search near the optimal individuals, and the local optimization capability of the later stage population is enhanced. u represents an influence factor, the size of the influence factor is changed along with the iteration number t, the image of u in the interval [0,1] is a parabola, and u is a variation strategy of which the parameter is used for controlling the differential evolution algorithm. u → 1, the variation mode can be inclined to DE/rand/1 to the maximum extent, and the algorithm has stronger global search performance so that the population individual diversity can be ensured. In the later stage u → 0, the mutation strategy tends to be DE/best/1, and the performance of the algorithm can reach better balance in the search capability, stability and optimizing speed, so that the low convergence speed of the algorithm in the early stage is avoided.

Step S205, selecting the components of the parent individuals or the variant individuals according to the cross factors to obtain cross individuals, and reversely selecting the components of the parent individuals or the variant individuals according to the cross factors to obtain anti-cross individuals;

and step S206, determining a moderate function according to the task completion time, the task execution cost and the load balance degree of the virtual machine.

The method is based on the characteristics of the cloud platform business model, achieves collaborative optimization of three targets of task completion time, execution cost and virtual machine load balancing, and converts a multi-target optimization problem into a single-target optimization problem for processing by using a weighted summation mode. Because there may be a large difference between the task completion time, the task execution cost, and the magnitude of the value of the three targets of the virtual machine load balancing, in order to balance the influence of the three targets on the adaptive function value, the normalization processing of the target value needs to be performed first, and the normalization processing of the target value in this embodiment is to take natural logarithms for the three values, and then design the appropriate function by using a weighted summation method. The formula is as follows:

f＝min[a ln(totalTime)+b ln(total Cost)+c ln(load)](31)

wherein f is an objective function of task scheduling in this embodiment, the objective in the scheduling process is to minimize the f value of an improved IADE moderate function of a differential evolution algorithm, a, b, and c in the formula are weight factors, and satisfy a condition that a + b + c is 1, and the weight factors are used to control the degree of influence on the comprehensive moderate value of the task completion time, cost, and resource load in the task scheduling process.

Step S207, obtaining the appropriate values of the target individual, the variant individual, the crossover individual and the anti-crossover individual according to the appropriate function, and obtaining the next generation individual according to the appropriate values, thereby decoding the next generation individual to obtain an initial scheduling result.

The individual selection step of the DE algorithm utilizes the concept of "win or lose" in nature. The individual fitness value is obtained according to a defined fitness function, and then greedy selection is carried out on the individual according to the value, if the fitness value is minimized, the smaller the fitness value is, the more excellent the individual is, therefore, when the fitness value of a new individual is smaller than that of the original individual, the original individual can be eliminated, the new individual can be reserved to the next generation, and therefore, the worst condition of the individual in the population is that the same fitness value as the original individual is kept, and the individual cannot be deteriorated. Therefore, the population individuals can be continuously optimized, and the convergence is stable finally.

Where f (x) is the objective function of the problem to be solved, the present embodiment assumes that the algorithm solves the minimization problem.

In the selection operation step of the standard DE algorithm, two individuals involved in comparison are target individuals and crossed individuals generated by the crossed operation, and the crossed operation further adjusts each dimension component of an individual vector obtained by the variation operation, so that the individuals are more diversified, and the algorithm can search an optimal solution in a larger range. However, a global optimal solution may be generated in the mutation operation step, and since the mutation operation is directly followed by the crossover operation on the variant individuals, the crossover operation replaces the local components of the mutation vectors, which may cause the excellent solution to be damaged.

In addition, in the cross operation step of the standard DE algorithm, a part of the components of the parent individuals or variant individuals are selected to form a cross individual vector according to the cross probability CR, and a different cross individual vector can be formed by performing reverse selection according to the cross probability CR, as shown in the bold part of fig. 7.

Therefore, in order to avoid destroying potentially superior individuals of the population, the algorithm is enabled to obtain the current optimal solution as much as possible. The standard DE algorithm is improved, and the selected operation step of the improved DE algorithm is that the variable vector v is selected_iCross vector u_iReverse crossing vector h_iAnd a target vector x_iAnd selecting the individual with the best fitness value to enter the next generation, wherein the improved selection strategy is shown as a formula (33):

in the selection operation step of the standard DE algorithm, when the moderate value of the crossover individual is larger than that of the parent individual, the parent individual is retained, which is improved by the present embodiment. When the vector of variation v_iCross vector u_iReverse crossing vector h_iIs compared with the parent target vector x_iWhen the moderate value of the algorithm is large, the average value of the optimal individual and the moderate value individual is taken as the next generation individual, so that the convergence speed and the population diversity of the algorithm are accelerated. The selection strategy in the formula (33) is further improved, and the formula of the finally improved selection strategy is as follows:

wherein x is_bestRepresents individuals with the best fitness value in the contemporary population, x_avgRepresents the median value, x, of all individuals in the contemporary population_avgIndividual component x_avg,jThe calculation formula is as follows:

according to the above improved modes of the mutation factor, the crossover factor, the mutation and the selection strategy, the algorithm pseudo code of the IADE of this embodiment is shown in table 2:

TABLE 2

The working process of the t iteration of the population in the IADE algorithm is shown in FIG. 8. The embodiment provides an improved differential evolution algorithm IADE by solving the problems that the standard DE algorithm is easy to be limited to local optimum, the later convergence speed is low, the parameter setting is difficult and the like, and the algorithm improves the strategy of a variation factor F, a cross factor CR, variation and selection of the standard DE algorithm. The IADE algorithm and DE algorithms of other variant modes are tested by 8 classical benchmark functions, and the results are compared through experiments to obtain the conclusion: the IADE algorithm is stronger than DE algorithms of other variant modes in convergence speed and the capability of jumping out of local optimum.

A Differential Evolution (DE) is a novel global optimization algorithm, and is widely applied due to its high efficiency, strong robustness, and simple implementation. The embodiment improves the defects that the convergence speed is slow, the parameter dependence is hard to set in the later iteration stage of the DE algorithm, and the like, and provides an improved DE algorithm (IADE), wherein the algorithm improves a scaling factor F, a cross probability CR and a variation and selection strategy of a standard DE algorithm, so that parameters can be adaptively adjusted along with the iterative Evolution of a population to meet different requirements of each period on parameter values, and the performance of the improved algorithm is verified through a benchmark function.

Since the load balancing problem is similar to the Q-value method (Q-value) seat allocation problem: the seat allocation problem aims to enable seats allocated to each party to be in proportion to the number of the seats allocated to each party and the number of the seats allocated to each party to be in proportion to the number of the seats per se, and the load balancing enables tasks allocated to each resource to be in proportion to the resource providing capacity. Therefore, the embodiment improves the standard Q value method according to the task scheduling problem, and applies the standard Q value method to the solution of the task scheduling load balancing problem.

By total population p_iProcessing capability VM representing a virtual machine VM_PjIn the present embodiment, the VM processing capability VM_PjThe size of the virtual machine is measured by the storage, calculation and bandwidth capacity of the virtual machine, and because the cost of executing the task is considered in the multi-objective optimization, the price per unit time of the VM is added. Synthesize above, VM_jIs expressed as:

number of currently assigned seats n in the formula of the standard Q-value method_iWith execution of tasks on the current VMM vmTime_jShowing that the seat number in the standard Q value method formula is changed into adding the task execution time taskTime migrated to the virtual machine_ij：

According to different improved Q value methods of processing capacity among different VMs, the improved Q value method formula is as follows:

and after clustering is completed in the preprocessing stage, a scheduling algorithm is started to be executed, task scheduling is performed in a plurality of task sets in parallel, and one task set corresponds to one resource set to form a small cloud computing environment. Tasks in the same category have similarity in resource requirements, and clustering also enables task allocation to have certain purposiveness. However, when a certain class of tasks of a user is too many, the corresponding resource set is overloaded, and other resource sets are overloaded and even idle, which also causes the problem of load imbalance of the whole system, and leads to the increase of task completion time, so that the resource utilization rate is not high, thereby reducing the user satisfaction and improving the operation cost of the cloud provider.

In the current task scheduling research based on clustering, the problem of load imbalance caused by clustering is not solved, the embodiment improves the problem, and after clustering and task allocation are completed, secondary rescheduling of local tasks is performed according to the load state of resources. In the first stage, the embodiment uses an improved differential evolution algorithm for scheduling, and takes resource load balancing, task completion time and execution cost as scheduling targets, and at this time, after scheduling is completed, the resource set is basically in a load balancing state. However, load imbalance may also exist among different resource sets, and secondary scheduling is performed to achieve load balance of the overall resources of the system so as to avoid the phenomena that the utilization rate of a CPU is too high but the utilization rate of a memory is low, or the utilization rate of the CPU is low but the utilization rate of the memory is too high. Therefore, global load balancing needs to be performed, that is, when a certain type of virtual machine still has a high-load state, the task of the virtual machine is scheduled to other types of low-load state resources for processing, so that the overall load of the system can reach a more balanced state.

Specifically, the implementation of load balancing local rescheduling in this embodiment can be mainly divided into three steps: firstly, grouping VMs according to the current load state of the VMs in the system; selecting tasks needing to be migrated from the heavy-load VM set, and placing the tasks into a task queue to be migrated; in the third step, the tasks in the task queue are rescheduled to the low-load VM to achieve global overall load balancing.

And step S208, dividing the virtual machines into a heavy load set, a balanced load set and a light load set according to the load values of the virtual machines in the initial scheduling result.

Calculating the current load state of each virtual machine after the primary task is distributed, and dividing the current load state into three queues according to the load state, wherein the three queues are respectively as follows: the system comprises an overload virtual machine queue, a balance virtual machine queue and a light load virtual machine queue. Tasks removed from an overloaded VM require finding a suitable VM that is too lightly loaded, since only such VM can be assigned to a new task. Virtual machines in the system are grouped in a manner that the adaptive threshold determines the virtual machine load status:

in the formula, load_maxLoad is the VM maximum load value_minFor VM minimum loadValue, load_midCalculated from maximum and minimum load values, load_upperFor overload boundary values, load_lowerThe light load boundary value.

Grouping the virtual machines according to a threshold value:

(1) the light load virtual machine VM- _ Over queue: load section_min，load_lower) The load of the representation task node is light, and more tasks can be borne.

(2) Equalizing the VM _ Normal queue: load section_lower，load_upper]The method shows that the load of the task node is optimal and the resource utilization condition is optimal.

(3) The overloaded virtual machine VM _ Light queue: load section (load)_upper，load_max]This indicates that the load on the task node is too heavy, and the amount of tasks to be allocated at this time is too large.

And step S209, selecting a task set needing rescheduling from the heavy load set to obtain a rescheduling task set.

In this embodiment, after the resources are grouped according to the load state, tasks in a task queue to be executed on the virtual machine in the heavy load state need to be selected, and then the selected tasks are rescheduled to the virtual machine in the light load state for execution, so that the virtual machine of the system after rescheduling is in a load balancing state.

In order to perform load balancing operation, a task list to be migrated of the overloaded VM needs to be determined first, and whether a load is present on the VM in a heavy load state or not is determined first_j-load_upperIf the tasks exist, the tasks are directly migrated out, the virtual machine can reach a load balancing state, if the tasks of the type do not exist, the tasks in the overload VM task queue are arranged in an ascending order from small to large according to the scale, the task with the smallest scale is added into the task list to be transferred each time, the task load is judged at the same time, if the tasks are still overloaded, the next task is added into the task list to be transferred, and recursion is carried out in sequence until the load of the VM is smaller than the load threshold load_upperAnd stopping at the upper limit.

And step S210, scheduling the tasks in the rescheduling task set to the virtual machines in the light load set by adopting a Q value algorithm.

Specifically, when a virtual machine is selected from a light load set, a Q value method based on task scheduling improvement is used for determining a proper virtual machine so as to clarify a target node of task transfer. And preferentially migrating the task to the virtual machine with a large Q value for execution.

And updating the load state of the heavy load and low load resource set VM after the task migration. After migration, if the resource load is in a moderate state, it is deleted from the heavily loaded resource set. And performing iterative load balancing rescheduling operation on the remaining virtual machines in the high load state until the resource set of the virtual machines in the low load state is empty.

EXAMPLE III

As shown in fig. 9, the multi-target task scheduling method of the third embodiment includes:

step 1: fuzzy clustering processing is respectively carried out on the independent tasks and the virtual machines by utilizing a fuzzy clustering algorithm, and the independent tasks and the virtual machines are divided into various classes according to task characteristics and virtual machine characteristics;

step 2: matching the sets according to the resource requirement of the task set and the comprehensive performance of the virtual machine set;

step 3: sequentially scheduling the tasks in the set to the virtual machine resources in the corresponding virtual machine set according to a multi-target task scheduling strategy of an improved differential evolution algorithm to obtain a scheduling result from the initial tasks to the virtual machines;

step 4: calculating the load state of the virtual machines, grouping the virtual machines according to the load values, and dividing the virtual machines into three groups: heavy, balanced and light loads;

step 5: and determining a task set to be redistributed, scheduling the overloaded task on the heavy-load virtual machine to the light-load virtual machine for running, and determining the target virtual machine in the rescheduling process by using a Q value method based on task scheduling improvement. After the task is redistributed, a final scheduling scheme from the task to the virtual machine is obtained;

step 6: and executing and completing the task.

The task scheduling policy MODE of this embodiment is specifically described as shown in table 3 below:

TABLE 3

In the embodiment, the optimization target of the embodiment is determined by considering the benefits of both sides of the user and the cloud service provider in the cloud platform and the actual condition of task scheduling, and the fuzzy clustering method is used for simultaneously clustering according to the characteristics of the tasks and the resources in the preprocessing stage, so that the exploration space of the tasks is reduced, and the task allocation speed is accelerated. By using the improved Q value method, the problem of unbalanced load caused by clustering among resource sets is solved, so that the whole cloud system reaches a relatively balanced load state, and the resource utilization rate is improved.

In order to test the effectiveness of the clustering and improved differential evolution algorithm-based multi-objective task scheduling strategy MODE for solving the cloud computing task scheduling problem, the embodiment verifies the MODE through a simulation experiment, and performs comparative analysis on the experiment results of the MODE and other scheduling algorithms.

In order to test the performance advantages of the clustering and improved differential evolution Algorithm-based multi-target task scheduling strategy in the embodiment in solving the cloud task scheduling problem, the embodiment compares the performance of the task scheduling strategy MODE provided by the embodiment with a Min-Min Algorithm, a standard DE Algorithm and a Genetic Algorithm (GA) through simulation testing.

The experimental environment of the task scheduling policy of this embodiment is shown in table 4:

TABLE 4

In this embodiment, data of each virtual machine and task used in the simulation experiment is randomly generated, and tables 5 and 6 show specific data value ranges:

TABLE 5

TABLE 6

Table 7 sets up the parameters for each algorithm:

TABLE 7

In this embodiment, the task scheduling policy provided by the present embodiment adopts three indexes, namely, task completion time (Makespan), Load balancing level (Load balancing level), and task execution Cost (Cost), to perform experimental comparison and evaluation.

(1) Clustering operation contrast experiment

When the task is scheduled, when the user task amount is large and the system resource amount is large, the time overhead of selecting the resource by the task is increased, so that the task scheduling efficiency of the cloud platform is low, and therefore, the resource selection range is necessarily reduced in the scheduling process, and the clustering operation is an effective means for reducing the selection range. In this embodiment, the execution time of the DE algorithm without clustering and the execution time of the DE algorithm after clustering are compared, and the comparison result is shown in fig. 10, where the horizontal axis represents the number of tasks in each scheduling and the vertical axis represents the execution time of the algorithm. From experimental data, clustering is obviously improved in execution time compared with a DE algorithm without clustering.

(2) Comparison experiment of task completion time

Task completion time (Makespan) is an index for evaluating throughput of heterogeneous cloud computing systems, and is used for measuring time spent from the beginning of a first task entering a cloud platform to the end of the execution of a last task.

Experiment one: different numbers of independent tasks are scheduled on 10 virtual machines by using a scheduling algorithm, in order to avoid the contingency of the experiment, the experiment statistics and comparison are repeated for 10 times respectively, and the experiment result is shown in fig. 11.

It can be seen from fig. 11 that the task completion time scheduled by each algorithm increases in a proportional manner as the number of tasks increases, while the number of virtual machines remains the same. When the task amount is small, the completion time of each algorithm task is not greatly different; when the task size becomes larger, a clear distinction is initially presented. Min-Min searches local optimization in a greedy way, while DE and GA algorithms both belong to evolutionary algorithms, and have obvious advantages in processing large-scale data combination optimization. The optimal solution approximation effect of the DE algorithm is more obvious compared with that of the GA algorithm, the GA algorithm controls the selection of individuals after individual intersection and variation according to the fitness value, the probability of selecting the individuals with small fitness values is increased when the minimization problem is processed, the DE algorithm variation vector is generated by random vector difference weighted combination of parent individuals, is intersected with the parent individuals to generate the crossed individual vector and is directly selected with the parent individuals, and only the optimal solution approximation effect is generated when the value of the DE algorithm is better than that of the parent individuals. Therefore, for the optimization problem, the DE algorithm has higher convergence speed and more accuracy than the GA algorithm, and the task completion time of the DE algorithm is superior to that of the GA algorithm. The standard DE algorithm is fixed in parameters, so that the search efficiency and the solving precision are relatively easily reduced, the parameters are improved for the DE algorithm in the MODE strategy, the self-adaptive change meets the requirements of each stage of population evolution, the mixed variation strategy is used for performing variation operation, the convergence speed of the algorithm is accelerated, the solving precision is improved, the time overhead of tasks on the matching virtual machine is reduced through clustering, and the method is more advantageous along with the increase of the task quantity.

Experiment two: scheduling 150 independent tasks on different numbers of virtual machines by using a scheduling algorithm, and respectively repeating 10 times of experiment statistical comparison to avoid the contingency of the experiment, wherein the experiment result is shown in fig. 12.

As can be seen from fig. 12, under the condition that the number of tasks remains unchanged, the task completion time of each algorithm decreases in an inverse manner as the number of virtual machines increases, and the number of visible resources greatly affects the task completion time. When the resources are sufficient, the task completion time reduction tendency becomes slow. The standard DE algorithm is slightly different from the GA algorithm, but the standard DE algorithm is slightly better, and the task completion time of the MODE is always better than that of the other comparative algorithms.

(3) Task completion cost comparison experiment

The profit of the cloud service provider can be measured by task execution Cost (Cost), and the smaller the Cost is, the higher the profit is.

The cost of task execution of each algorithm is compared, and in order to avoid the contingency of the experiment, the statistical comparison of the experiment is repeated for 10 times, and the experimental result is shown in fig. 13.

It can be seen from fig. 13 that the task cost increases as the number of tasks increases. When the number of tasks is small, the performance gap between the standard DE, GA and MODE algorithms is not large, and when the number of tasks is increased, the task cost of MODE is obviously lower than that of other algorithms. The Min-Min algorithm is significantly more expensive to perform than other algorithms because the Min-Min greedy property always assigns tasks to more powerful virtual machines to seek time optimization, but the cost overhead of such virtual machines can be large.

(4) Contrast experiment of load balance

Load balancing level (Load balancing level) to measure the system resource utilization. The task scheduling algorithm must fully consider the load balance among the nodes when performing task allocation, so that the load of each node is balanced as much as possible, and the task is prevented from waiting to run in the nodes with better performance.

And comparing the load of the VMs, and measuring the overall load balancing condition of the system resources by the standard deviation of the task execution time of each VM in the experiment. In order to avoid the contingency of the experiment, the statistical comparison of the experiment was repeated 10 times, and the experimental results are shown in fig. 14.

Fig. 14 shows the magnitude of the load balancing value when the virtual machine loads different numbers of tasks for scheduling, and it can be seen from the figure that as the number of tasks increases, the load of the virtual machine increases, and because the Min-Min algorithm only takes the execution time of the tasks as an optimization target, neglecting the consideration of resource load balancing, the load characteristic is the worst. The standard DE algorithm is slightly better than the GA algorithm in scheduling, and the standard DE algorithm is lower in solving precision and speed than the IADE algorithm due to the fixed parameters and the limitation of a single variation strategy, so that the MODE scheduling strategy is slower in load increase speed and better in load value compared with other algorithms.

As can be known from performance verification of the algorithm and comparative analysis of the algorithm and other algorithms, for the problem of independent task scheduling in the cloud environment, the strategy MODE of the embodiment can not only improve the execution efficiency of the task, shorten the completion time, but also reduce the cost and expense of task execution, so that the system keeps higher resource utilization rate.

In the embodiment, the task scheduling policy MODE provided by the embodiment is tested through a simulation experiment, and is compared with a standard DE algorithm, a GA algorithm and a Min-Min algorithm through the simulation experiment, and the comparison experiment result shows that the MODE policy algorithm provided by the embodiment has good performance in completion time, load balance and execution cost.

Referring to fig. 15, the multi-target task scheduling system provided in the embodiment of the present invention includes:

a memory 10, a processor 20 and a computer program stored on the memory 10 and executable on the processor 20, wherein the processor 20 implements the steps of the multi-objective task scheduling method proposed by the present embodiment when executing the computer program.

The specific working process and working principle of the multi-objective task scheduling system in this embodiment may refer to the working process and working principle of the multi-objective task scheduling method in this embodiment.

The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes will occur to those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-objective task scheduling method, the method comprising:

respectively carrying out fuzzy clustering processing on tasks and virtual machines by using a fuzzy clustering algorithm to obtain a task cluster set and a virtual machine cluster set, and matching the task cluster set with the virtual machine cluster set to obtain a matching set, wherein the matching of the task cluster set with the virtual machine cluster set comprises the following steps: according to the computing capacity, the bandwidth and the storage capacity of the virtual machines in the virtual machine cluster set, obtaining the resource performance of the virtual machines through weighted computing, according to the average value of the resource performance of all the virtual machines in the virtual machine cluster set, obtaining the comprehensive resource performance of the virtual machine cluster set, according to the length, the size and the output data size of tasks in the task cluster set, obtaining the resource requirements of the tasks through weighted computing, according to the average value of the resource requirements of all the tasks in the task cluster set, obtaining the comprehensive resource requirements of the task cluster set, and according to the distance between the comprehensive resource performance of the virtual machine cluster set and the comprehensive resource requirements of the task cluster set, matching the task cluster set and the virtual machine cluster set to obtain a matching set;

scheduling the tasks in the matching set to the virtual machines in the matching set by adopting a differential evolution algorithm to obtain an initial scheduling result, wherein the scheduling of the tasks in the matching set to the virtual machines in the matching set by adopting the differential evolution algorithm to obtain the initial scheduling result comprises the following steps: adopting a one-dimensional group to encode individuals of a population in a differential evolution algorithm according to the number of tasks, the number of virtual machines and the number of tasks in the matching set, calculating a variation factor and a cross factor according to a current evolution algebra, carrying out variation operation on population individuals according to the variation factor and combining various variation strategies to obtain variation individuals, selecting components of parent individuals or the variation individuals according to the cross factor to obtain cross individuals, reversely selecting the components of the parent individuals or the variation individuals according to the cross factor to obtain reverse cross individuals, determining a moderate function according to task completion time, task execution cost and load balance of the virtual machines, obtaining the moderate values of target individuals, the variation individuals, the cross individuals and the reverse cross individuals according to the moderate function, and obtaining next generation individuals according to the moderate values, therefore, the next generation individual is decoded to obtain an initial scheduling result, and the specific formula for calculating the variation factor and the cross factor according to the current evolution algebra is as follows:

wherein F (T) represents a mutation factor, CR (T) represents a cross factor, w (T) represents a weight factor, T represents an evolution algebra, T represents a population maximum evolution algebra, F (T) represents a mutation factor, CR (T) represents a cross factor, w (T) represents a weight factor, T represents a population maximum evolution algebra_maxDenotes the maximum value of the variation factor, F_minRepresents the minimum value of the variation factor, CR_minDenotes the minimum value of the crossing factor, CR_maxRepresents the maximum value of the crossing factor;

selecting a task set needing to be rescheduled from the heavy load set to obtain a rescheduled task set;

2. The multi-objective task scheduling method according to claim 1, wherein a variety operation is performed on population individuals according to the variation factors and in combination with a plurality of variation strategies, and a specific formula for obtaining a variant individual is as follows:

wherein λ represents a weight factor, u represents an influence factor, v_i(T) represents the variant individual corresponding to the current evolutionary algebra of the population, T represents the evolutionary algebra, T represents the maximum evolutionary algebra of the population,x_irepresenting the current individual, x_r1(t)、x_r2(t)、x_r3(t)、x_r4(t) represents four differences from x corresponding to the current evolutionary algebra of the randomly selected population_iIndividual of (1), x_best(t) represents the best individual in the contemporary population, and F represents the variation factor.

3. The multi-objective task scheduling method according to claim 2, wherein the calculation formula for obtaining the fitness values of the objective individuals, the variant individuals, the crossover individuals and the anti-crossover individuals according to the fitness function is specifically as follows:

where f denotes a moderating function, and f ═ min [ a ln (totalTime) + b ln (total Cost) + c ln (load)]，x_i ^t+1Representing the next generation of target individuals, u_i ^t+1Represents the next generation of crossover individuals, f (u)_i ^t+1) Representing the fitness of the next generation of crossover individuals, v_i ^t+1F (v) representing the next generation of variant individuals_i ^t+1) Represents the moderate value of the next generation variant individual, h_i ^t+1Representing the next generation of anti-crossover individuals, f (h)_i ^t+1) Representing the fitness of the next generation of anti-crossover individuals, f (x)_i ^t) Representing the fitness value, x, of the current generation of target individuals_bestRepresents individuals with the best fitness value in the contemporary population, x_avgThe method comprises the steps of representing the median of all individuals in the current generation population, wherein totalTime represents task completion time, total Cost represents task operation Cost overhead, load represents load balance, a, b and c are weighting factors, and the condition that a + b + c is 1 is met.

4. The multi-target task scheduling method according to any one of claims 1 to 3, wherein a calculation formula of a Q value in the Q value algorithm is specifically:

wherein Q is_iDenotes the Q value after assignment of the task to the jth virtual machine, vm_pjRepresenting the processing power of the jth virtual machine in the lightly loaded set, vmTime_jRepresents the execution time, taskTime, of the task on the jth virtual machine in the light load set_ijAnd representing the task execution time for migrating the ith task in the reschedule task set to the jth virtual machine in the light load set.

5. The multi-objective task scheduling method of claim 4, wherein:

maximum value of the variation factor F_max1, minimum value of said variation factor F_min0.4, the minimum value of the crossing factor is CR_min0.6, the maximum value of the crossing factor is CR_max＝0.9。

6. A multi-objective task scheduling system, the system comprising:

memory (10), processor (20) and computer program stored on the memory (10) and executable on the processor (20), characterized in that the steps of the method according to any of the preceding claims 1 to 5 are implemented when the computer program is executed by the processor (20).