CN104536804A - Virtual resource dispatching system for related task requests and dispatching and distributing method for related task requests - Google Patents

Abstract

The invention discloses a virtual resource dispatching system for related task requests and a dispatching and distributing method for the related task requests. Operation performance can be effectively guaranteed when the communication related tasks exist, and meanwhile an effective balance point is selected from a system load. In addition, by the adoption of dominant_share evaluation features based on a DRF, it can be prevented that too many resources are distributed, and fragments of partial resources are generated. By the adoption of a distributing strategy based on greed, compared with a linear programming algorithm, the distributing algorithm has a great time performance advantage, the dispatching time delay is reduced, and good user experience is obtained. Finally, the dispatching algorithm can improve the resource utilization condition of a platform, reduce the number of available tasks in waiting, increase the resource utilization rate of the cloud computing platform, and improve the performance of the platform and the overall processing performance of the platform.

Description

Towards the schedule virtual resources system of associated task request and scheduling and distribution method

Technical field

The invention belongs to computer science, relate to schedule virtual resources system under a kind of cloud computing environment, specifically a kind of schedule virtual resources system towards associated task request and scheduling and distribution method.

Background technology

The computation schema of its employing innovation of cloud computing makes user obtain almost limitless computing power and rich and varied information service at any time by internet, and the business model of its innovation makes user can take freely calculating and service, pay according to quantity.Current cloud computing has been merged with virtual, Service Management robotization and has been standardized as innovating technologies in a large number of representative.Cloud computing, by the retractility of Intel Virtualization Technology and dirigibility, improves resource utilization, simplifies administering and maintaining of resource and service; Utilize information service automatic technology, by Resource Encapsulation be service offering to user, decrease the operation cost of data center; Utilize standardization, facilitate exploitation and the payment of service, shorten the on-line time of customer service.And the resources of virtual machine scheduler of cloud computing or positive dispatching algorithm are the important module of cloud computing platform, be solve Resource Fusion efficiently, shared important composition.

Mainly contain the method that following several algorithm solves the resources of virtual machine scheduling strategy of IaaS cloud computing platform at present:

1. from the angles of loading of platform, when individual task or application send the request of resources of virtual machine, platform checks the surplus resources in all physical machine, the physical machine of the resource meeting physical machine request is filtered out, and makes the load balancing in each physical machine as far as possible.

2. from the efficiency of the utilization of the resource of platform, when task or application send resources of virtual machine request, surplus resources on inspection belongings keeps house, minimum physical machine is adopted to meet task or the request of application to resource, what make task platform and integrally can have less use physical resource, reduces the unnecessary wasting of resources.

3., based on the dispatching algorithm of fairness, for the fair scheduling algorithm of DRF algorithm, main each Resourse Distribute of consideration is directly balanced with rationally, can make the Pareto optimality reached the request of resource and distribution between task in economics.

The above-mentioned kind of method that method is solving resources of virtual machine scheduling problem in cloud computing has respective limitation:

1. load-balancing algorithm is consistent as far as possible in order to ensure the virtual machine load in each physical machine, the allocative decision of virtual machine can be caused relatively loose, cause having the mutual or task of a large amount of network communication in a large number or application can maybe worse network condition, cause the decline of service quality or the time delay of tasks carrying increases.

2. the algorithm of platform utilization factor, adopt a small amount of physical machine meet the resources of virtual machine request meeting of application or obtain reasonable resource utilization as far as possible, but this algorithm is a typical np problem, solve this problem and often cause higher scheduling cost, therefore seldom adopt in the cloud computing platform of reality.

This utilization power in resource of 3.DRF algorithm can cause many fragmentation figures, cause distributing, schedulable number of times is not enough to some extent, and in addition owing to have ignored the relation of the position of physical machine own, the network condition that can cause the actual motion of physical machine resource and task is unmatched.

To sum up, the consideration that above-mentioned three kinds of algorithms all exist in the problem of incidence relation in the communication request of task and tasks carrying is not enough all to some extent, and all there is oneself not enough factor itself to the situation of the request dispatching of resource separately.So often good performance advantage can not be obtained to the scheduling result of the task requests of mass communication.

Summary of the invention

The object of the invention is to solve the problem, the schedule virtual resources system towards associated task request under a kind of cloud computing platform and scheduling and distribution method are provided, the method, by the reasonable distribution of virtual machine, meets and there is the performance requirement of the task of incidence relation and the characteristic of cloud computing platform load balancing.

To achieve these goals, the technical solution adopted in the present invention comprises the following steps:

Towards a schedule virtual resources system for associated task request, comprise resource management controller, resource request resolver and scheduling of resource iterator;

The resource situation of physics host of resource management controller for receiving resource monitoring and obtaining;

Resource request resolver is used for front end pretreatment process and resource request encapsulates;

Scheduling of resource iterator, for carrying out the iterative control process dispatched, produces scheduling of resource result.

Described resource management controller specifically for:

1) dock with monitoring resource interface, Real-time Obtaining cloud computing cluster resource state status;

2) in conjunction with scheduling of resource iteration controller, extract the resource situation of each physical machine in platform from monitor data, resource situation comprises the corresponding state of cpu, internal memory, bandwidth and hard disk;

3) according to the resource situation of physical machine each in cloud computing cluster, when the controller request of scheduling iteration device, real-time calculate total resources in cluster and surplus resources total amount, calculate dominant_share and dominant_desire of each physical machine under current state simultaneously;

4) under concurrent environment, ensure the consistance of data, ensure the consistance of cluster state information; When the high concurrent operation of scheduling of resource iterator, provide the consistance to resource, atomicity and transactional.

The responsible following content that described resource request resolver is concrete:

1) receive the request json file of the task scheduling request user that framework is sent, extract the cpu that scheduling iteration device is concerned about, the resource request such as internal memory, simultaneously the resource request of further encapsulation user;

2) analytical framework task association request, runs strong UNICOM component, the graph-theoretical algorithms such as topological sorting, produces the task sequence with special marking;

3) provide the encapsulation operation of union operation, in the operation utilizing Union-find Sets, provide follow-up scheduling of resource iterator to carry out operating to carry out union operation, further scheduling of resource iterator can be inquired about and merge history generation scheduling of resource result.

Described scheduling of resource iterator comprises scheduling of resource iteration controller, scheduling of resource iterative combiner, scheduling of resource iterative sequencing device and scheduling of resource iterative estimation device;

Scheduling of resource iteration controller, for carrying out the iteration control of dispatching, controls scheduling of resource scheduling iteration combiner and the collaborative work of scheduling of resource sorting unit, judges stopping criterion for iteration; And in the iterative process of iteration, resource request is distributed, carry out the calculating of the resource of actual distribution;

Scheduling of resource iterative combiner is responsible for the operation carrying out merging request according to the design of CBDRF algorithm, according to the algorithm of CBDRF design, adopts the method for Union-find Sets, carries out union operation according to the markd task sequence of band that resource request resolver sends;

Scheduling of resource iterative sequencing device is used for auxiliary resources scheduling iteration controller in the process of carrying out Resourse Distribute and carries out Resourse Distribute;

Scheduling of resource iterative estimation device, for assessment of scheduling result, records current assessment result; The scheduling intermediate result simultaneously produced with previous iteration as calculated compares, if current assessed value is better than previous scheduling result, recording dispatching result is current results, otherwise give up current result, notice scheduling of resource iteration controller carries out next round iterator.

Towards schedule virtual resources and the distribution method of associated task request, comprise the following steps:

A1 resource management controller obtains the resource status of cloud computing platform, carries out corresponding pretreatment operation;

A2 resource request resolver connects resolves task status according to the correlation model of graph theory, generates corresponding task sequence;

A3 scheduling of resource iterator carries out the assigning process based on max_min operation, generates iteration scheduling result;

A4 scheduling of resource iterative estimation device assessment result, preserves the optimal value calculated;

A5 carries out the union operation of associated task according to the model of graph theory, merges corresponding task requests;

Can A6 scheduling of resource iteration controller judge distribute, if can go to step 3), otherwise Output rusults.

In described steps A 1 to A4, the dispatching distribution strategy of scheduling of resource iteration control implement body carries out in accordance with the following steps:

B1 resource resolver resolves user ask, be each request initialization request example, each example be five-tuple wherein:

Five-tuple comprises cpu, mem, related_request, merge_history, scheduled_host; Cpu represents the cpu check figure of the virtual machine of task requests; Mem represents the virutal machine memory size of task requests, in units of GB; Related_request represents associated task request set; Merge_history represents merging history, and scheduling iteration device resolves final scheduling result in conjunction with scheduled_host from merging history; Scheduled_host represents the scheduling result of request, is resolved the scheduling result of final request by merge_history;

The initial value of above-mentioned cpu, mem, related_request obtains from the request of user, merge_history and scheduled_host just initial set is empty set;

B2 is abstracted into a little by there being mutually the task of interchange, and network service relation between them is abstracted into limit, by related_request, request analysis is abstracted into a graph of a relation; Run the strong UNICOM component of graph-theoretical algorithm calculation task graph of a relation by resource resolver, be marked in request example;

B3request example sends resolution data to scheduling of resource iterator, and simultaneously scheduling of resource iterator sends request to resource management controller, with obtain cloud computing platform real time resources state;

B4 resource management controller is that each physical machine creates host example, and each example is four-tuple, wherein:

Four-tuple comprises cpu_total, cpu_used, mem_total, mem_used; Cpu represents that single physical machine represents cpu check figure respectively; Cpu_used represents the cpu check figure that single physical machine has used; Mem_total represents single physical machine internal memory sum, in units of GB; Mem_uesd represents that single physical has used internal memory sum, in units of GB;

Above-mentioned numerical value all obtains these numerical value by the monitor-interface of cloud computing platform;

B5 using the request of user as the resource pool that will distribute, by abstract for these resource request total resources for distributing to physical machine when multiple resource request arrives time, the capacity of physical machine is calculated the dominant_share of each physical machine host as the request vector of reality, in resource vector, maximum resource occupation rate is called the dominant_share of physical machine simultaneously; Maximal value in surplus resources vector becomes the dominant_desire of this physical machine; Its computing formula is as follows:

The total number vector of cloud computing resources:

R＝<r ₁,…r _m>

Single physical machine i total resources:

HT _i＝<ht _i,1,…ht _i,m>

Single physical machine i is Resources allocation total amount:

HU _i＝<hu _i,1,…hu _i,m>

The dominant_share computing method of single physical machine i:

${\mathrm{HS}}_i={\max }_{j=1}^m\{h{u}_{i,j}/r_j\}$

The dominant_desire computing method of single physical machine i:

${\mathrm{HD}}_i={\max }_{j=1}^m\{h{t}_{i,j}-h{u}_{i,j}\}$

B6 resource iterative combiner calculates dominant_share and dominant_desire of request, and according to the ascending sequence of the size of dominant_share, identical dominant_share then sorts according to the size of dominant_desire is ascending; Wherein dominant_share and dominant_desire of request is as described below:

Task requests total number resource vector:

T＝<t ₁,…t _m>

Task requests single physical machine i request resource total amount:

TR _i＝<tr _i,1,…tr _i,m>

The dominant_share computing method of task requests physical machine i:

${\mathrm{TS}}_i={\max }_{j=1}^m\{t{r}_{i,j}/t_j\}$

The dominant_desire computing method of task requests single physical machine i:

${\mathrm{TD}}_i={\max }_{j=1}^m\left\{t{r}_{i,j}\right\}$

B7 scheduling of resource iterative sequencing device and scheduling of resource iterator computing greedy algorithm;

B8 scheduling of resource iterative estimation device utilizes the fair index of Jain ' s fairness index: Jian Shi, the dominant_share vector T S (ts_1 of computational resource request request, ts_2, ts_k), cloud computing platform physical machine vector dominant_share vector HS (hs_1, hs_2 ... hs_n), Value=J (TS)/J (HS) now and allocative decision is recorded; Wherein the computing method of the fair index of Jian Shi are as described below:

$J(x_1,x_2,...,x_n)=\frac{{\left({\mathrm{\&Sigma;}}_{i=1}^n{x}_i\right)}^2}{n*{\mathrm{\&Sigma;}}_{i=1}^n{x}_i^2}$

B9 scheduling of resource iterative estimation device calculates Value=J (TS)/J (HS), if the optimal value before being greater than, records the solution of this formula;

B10 scheduling of resource iterative combiner is chosen in the request sorted and is carried out association union operation from task strong continune is as calculated heavy, thus renewal merging merge_history skips to step B5, otherwise skips to step B11;

B11 algorithm terminates, and exports solution.

In described steps A 2, the concrete grammar of resolving task is:

Incidence relation between C1 extraction task, is described as the structure of a figure by the incidence relation of task;

C2, according to the relation of graph theory, calculates strong UNICOM component, marks each task and belongs to which UNICOM's component;

By each strong UNICOM component extraction out, its assigned sequence sorts according to resource request number is ascending C3; The strong UNICOM component obtained is regarded as an overall task, and by the model of figure simultaneously, in conjunction with original relation, obtain a new task incidence relation; By original complex task figure boil down to DAG task image;

DAG task image is carried out topological sorting by C4, obtains a topological sequences of new figure, and in each result of topological sequences, if having the new point that UNICOM's component point reduction causes, the sequence that this some step C3 obtains is replaced.

In described steps A 3, the concrete calculation process based on the assigning process of max_min operation carries out in accordance with the following steps:

D1 calculates dominant_share and dominant_desire of all task requests, and sorts according to its size;

D2 calculates the vector of dominant_share and dominant_desire of All hosts; Choose wherein first task, select the main frame that dominant_share and dominant_desire of the main frame that can distribute is minimum;

If D3 becomes the distribution of work, upgrade the vector of dominant_share and dominant_desire of All hosts;

If D4 has residue, main frame is also unallocated, continue above-mentioned steps D3, otherwise assigning process terminates.

In described steps A 5, associated task union operation carries out as follows according to situation:

1) if previous task and rearmounted task are communication association relations, so all must being met of two task resource requests during the merging of these two tasks to the request of resource, in this situation situation, the amalgamation result of resource is the linear superposition of 2 task resource requests;

2) if previous task and rearmounted task are DAG incidence relations, so the resource request of these two tasks is DAG relations; If these two tasks are not merged history, so compare the size cases of previous task and rearmounted task, retain larger;

3) if merged task when having in fruit previous task and rearmounted task, so merge according to following rule: be set to later merging for example, now the resource request of previous task may candidate item as one of rearmounted task resource request; Therefore, when merging, need the history of the request traveling through rearmounted task merging, investigate the history of rearmounted task merging;

4) if there is the task requests sequence that every resource requirement is all less than preposition resource request in the resource requirement of rearmounted historic task, in rearmounted history, so ignore a rearmounted resource request wherein, the result of merging be the request vector of rearmounted task with the difference of the resource vector ignored and preposition resource requirement with;

5) if when the existence of preposition resource is less than a certain item task of rearmounted resource, ignore the resource requirement of this preposition task;

6) if not above-mentioned several situation, then merge according to 2 communication association tasks.

Compared with prior art, the present invention has following beneficial effect:

The present invention effectively can ensure the operational performance that there is communication association task, on system load, meanwhile choose an effective equilibrium point.In addition, the present invention is based on DRF algorithm and realize, the assessment of dominant_share, the generation of the distribution of excess resource and the fragment of part resource can be prevented.The present invention adopts the allocation strategy based on greed, and allocation algorithm has larger time performance advantage compared with linear programming relax, reduces scheduling delay, obtains good Consumer's Experience.Finally, dispatching algorithm of the present invention can the resource utilization of lifting platform, reduces the number of tasks of idle waiting, promotes the resource utilization of cloud computing platform and the performance of platform, improve the bulk treatment performance of platform further.

Accompanying drawing explanation

Fig. 1 is module composition and the processing flow chart of Resource Scheduler of the present invention;

Fig. 2 is that Resource Scheduler of the present invention calculates scheduling flow figure;

Fig. 3 is fairness index contrast figure of the present invention;

Fig. 4 is the cpu request comparison diagram that the present invention resolves task;

Fig. 5 is the memory request comparison diagram that the present invention resolves task;

Fig. 6 is the comparison diagram of real time execution task situation of the present invention;

Fig. 7 is the comparison diagram of idle waiting task situation of the present invention;

Fig. 8 is the comparison diagram of Runtime of the present invention.

Embodiment

Below in conjunction with drawings and Examples, the present invention is further detailed explanation.

See Fig. 1 and Fig. 2, resource scheduling system of the present invention, comprises resource management controller, resource request resolver, scheduling of resource iterator.

Described explorer is mainly used in the resource situation receiving the physics host that resource monitoring obtains, and the operation related to mainly contains following:

1) dock with monitoring resource interface, Real-time Obtaining cloud computing cluster resource state status.

2) in conjunction with scheduling of resource iteration controller, from monitor data, extract the resource situation of each physical machine in platform, comprise cpu, internal memory, bandwidth, the corresponding state of hard disk.

3) according to the resource situation of physical machine each in cloud computing cluster, when the controller request of scheduling iteration device, real-time calculate total resources in cluster and surplus resources total amount, in scheduling of the present invention, explorer also should calculate dominant_Share and dominant_desire of each physical machine under current state.

4) under concurrent environment, ensure the consistance of data, ensure the consistance of cluster state information.When the high concurrent operation of resource scheduling system iterator, provide the consistance to resource, atomicity and transactional.

In dispatching system of the present invention, the front end pretreatment process that resource request resolver is mainly important and resource request encapsulate.Because scheduling of resource iteration controller needs to operate accordingly resource request, resource request resolver needs the resource request of the task to different frames to operate, and this just requires the operation of the necessary encapsulation resource request of resource request resolver.In addition, for the parsing of associated task, resource request resolver is responsible for processing the parsing between associated task, need for onrelevant task, DAG task, design corresponding algorithm with communication task to resolve, the result of final parsing is submitted to scheduling of resource iteration controller with the form of the sequence with special marking.Specifically, the following aspects is responsible for by scheduling resolver:

1) receive the request json file of the task scheduling request user that framework is sent, extract the cpu that scheduling iteration device is concerned about, the resource request such as internal memory, simultaneously the resource request of further encapsulation user.

2) analytical framework task association request, runs strong UNICOM component, the graph-theoretical algorithms such as topological sorting, produces the task sequence with special marking.

3) provide the encapsulation operation of union operation, in the operation utilizing Union-find Sets, provide follow-up scheduling of resource iterator to carry out operating to carry out union operation, further scheduling of resource iterator can be inquired about and merge history generation scheduling of resource result.

Scheduling iteration device is the nucleus module of dispatching system of the present invention.Scheduling of resource iterator mainly carries out the iterative control process dispatched, and produces scheduling of resource result.Scheduling iteration device comprises scheduling of resource iteration controller further, scheduling of resource iterative combiner, scheduling of resource iterative sequencing device, scheduling of resource iterative estimation device.Four modules.

Scheduling of resource iteration controller

Scheduling of resource iteration controller is the important control element of scheduling of resource iterator, mainly carries out the iteration control of dispatching, and is the important process module of operation dispatching algorithm.Iteration controller controls scheduling of resource scheduling iteration combiner and the collaborative work of scheduling of resource sorting unit, judges stopping criterion for iteration.And scheduling of resource iteration controller is in the iterative process of iteration, need to distribute resource request, carry out the calculating of the resource of actual distribution.The main operation that scheduling of resource iteration controller relates to has following:

1) receive the request of resource request resolver, control the initial work of scheduling of resource iterator.Control scheduling of resource iterative sequencing device, carry out the assessment sequence of physical machine according to dominant_share and dominant_desire.

2) scheduling of resource iterator carries out scheduling controlling, according to greedy algorithm, takes scheduling controlling process, adopts max_min method, carries out greed distribute dominant_share

3) call scheduling of resource iterative combiner, the carrying out controlling the request of the same connected component parsed merges.

4) the scheduling iteration number of times of Control and Schedule iterator, and judge whether the result merged can carry out next round iteration.During iteration ends, return final scheduling result.

Scheduling iteration combiner

The union operation of scheduling iteration combiner is the important step in scheduling iteration operation.In order to produce the scheduling intermediate result of intermediate iteration.Actual in the process of scheduling, the operation carrying out merging request according to the design of CBDRF algorithm is responsible for by scheduling iteration combiner, according to the algorithm of CBDRF design, adopt the method for Union-find Sets, carry out union operation according to the markd task sequence of band that resource request resolver sends.

Scheduling iteration sorting unit

Scheduling iteration sorting unit is the auxiliary link in the operation of scheduling iteration.The Main Function of this sorting unit is that auxiliary resources scheduling iteration controller carries out Resourse Distribute in the process of Resourse Distribute.Distribute in order to greed can be carried out to domainant_share, must sort respectively to physical machine and resource scheduling system, sort according to different domainant_share and domainant_desire, in order to accelerate this sequencer procedure, the union operation of scheduling iteration sorting unit to resource have employed to pile for key data structure sorts.Because in each iterative process, all cloud computing resource scheduling method and the systems entering task based access control of the present invention association and load balancing with virgin state to the sequence of the operation of physical machine, by abstract for the resource request of all tasks be a node in figure, the request that analytical Calculation goes out same connected component is carried out by resource request resolver, in the process of scheduling of resource iteration, the task of the request of the same connected component of continuous merging, to obtain good performance advantage, assess the loading condition of current cloud computing platform simultaneously, choose best equilibrium point in both.So in the maintenance of sorting unit, remain the copy of sequence of resource information state on original physical machine, with the carrying out accelerating whole scheduling.

Scheduling iteration evaluator

Scheduling iteration evaluator is last link of scheduling iteration device, is also an important module of scheduling of resource iterator.This module in order to assess the fairness of scheduling result, load equilibrium, etc. factor.Support the interpolation of multiple valuation functions.Realize in this this scheduling, have employed the fair evaluation number based on Jian Shi coefficient.In addition, scheduling of resource iterative estimation device is except commenting the carrying out of scheduling result, record outside current assessment result, another vital role of scheduling of resource iterative estimation device is that the scheduling intermediate result produced with previous iteration as calculated compares, if current assessed value is better than previous scheduling result, recording dispatching result is current results, otherwise give up current result, notice scheduling of resource iteration controller carries out next round iterator.

Further, on the basis of above-mentioned module, resource scheduling system of the present invention is dispatched according to following step.

A1 resource management controller obtains the resource status of cloud computing platform, carries out corresponding pretreatment operation.

A2 resource request resolver connects resolves task status according to the correlation model of graph theory, generates corresponding task sequence

A3 scheduling of resource iterator carries out the assigning process based on max_min operation, generates iteration scheduling result.

A4 scheduling of resource iterative estimation device assessment result, preserves the optimal value calculated.

A5 carries out the union operation of associated task according to the model of graph theory, merges corresponding task requests.

Can A6 scheduling of resource iteration controller judge distribute, if can A3 be gone to step, otherwise Output rusults.

Resource scheduling system workflow

Based on a resources of virtual machine scheduling scheme for task incidence relation under cloud computing platform, for generation of the dispatching distribution strategy of virtual machine, specifically comprise following steps

B1 resource resolver resolves user ask, be each request initialization request example, each example be five-tuple (cpu, mem, related_request, merge_history, scheduled_host) wherein:

Cpu represents the cpu check figure of the virtual machine of task requests.

Mem represents the virutal machine memory size of task requests, in units of GB.

Related_request represents associated task request set.

Merge_history represents merging history, and scheduling iteration device resolves final scheduling result in conjunction with scheduled_host from merging history.

Scheduled_host represents the scheduling result of request, is resolved the scheduling result of final request by merge_history.

The initial value of above-mentioned cpu, mem, related_request obtains from the request of user, merge_history and scheduled_host just initial set is empty set.

Owing to there is network service mutually between associated task in B2, same physical machine is placed two virtual machines that there is network service can elevator system performance, and the utilization ratio of resource can be promoted, therefore will the task of interchange be had mutually to be abstracted into a little, network service relation between them is abstracted into limit, by related_request, request analysis is abstracted into a graph of a relation.The strong UNICOM component of graph-theoretical algorithm calculation task graph of a relation is run by resource resolver.Be marked in request example.

B3request example sends resolution data to scheduling of resource iterator, and simultaneously scheduling of resource iterator sends request to resource management controller, with obtain cloud computing platform real time resources state.

B4 resource management controller is that each physical machine creates host example, each example be four-tuple (cpu_total, cpu_used, mem_total, mem_used) wherein:

Cpu represents that single physical machine represents cpu check figure respectively.

Cpu_used represents the cpu check figure that single physical machine has used.

Mem_total represents single physical machine internal memory sum, in units of GB.

Mem_uesd represents that single physical has used internal memory sum, in units of GB.

Above-mentioned numerical value all obtains these numerical value by the monitor-interface of cloud computing platform.

B5 using the request of user as the resource pool that will distribute, by abstract for these resource request total resources for distributing to physical machine when multiple resource request arrives time, the capacity of physical machine is calculated the dominant_share of each host (physical machine) as the request vector of reality, the maximal value in the occupation rate of the resource had for resource vector claims the dominant_share. of physics household which owns machines for production to calculate dominant_desire resource vector remains maximum resources simultaneously simultaneously.Shown in it is defined as follows:

The total number vector of cloud computing resources:

R＝<r ₁,…r _m>

Single physical machine i total resources:

HT _i＝<ht _i,1,…ht _i,m>

Single physical machine i is Resources allocation total amount:

HU _i＝<hu _i,1,…hu _i,m>

The dominant_share computing method of single physical machine i:

${\mathrm{HS}}_i={\max }_{j=1}^m\{h{u}_{i,j}/r_j\}$

The dominant_desire computing method of single physical machine i

${\mathrm{HD}}_i={\max }_{j=1}^m\{h{t}_{i,j}-h{u}_{i,j}\}$

B6 resource iterative combiner calculates dominant_share and dominant_desire of request, and according to the ascending sequence of the size of dominant_share, identical dominant_share then sorts according to the size of doaminant_desire is ascending.Wherein dominant_share and dominant_desire of request is as described below:

Task requests total number resource vector:

T＝<t ₁,…t _m>

Task requests single physical machine i request resource total amount:

TR _i＝<tr _i,1,…tr _i,m>

The dominant_share computing method of task requests physical machine i:

${\mathrm{TS}}_i={\max }_{j=1}^m\{t{r}_{i,j}/t_j\}$

The dominant_desire computing method of task requests single physical machine i

${\mathrm{TD}}_i={\max }_{j=1}^m\left\{t{r}_{i,j}\right\}$

B7 scheduling of resource iterative sequencing device and scheduling of resource iterator computing greedy algorithm.

B8 scheduling of resource iterative estimation device utilizes the fair index of Jain ' s fairness index: Jian Shi, the dominant_share vector T S (ts_1 of computational resource request request, ts_2, ts_k), cloud computing platform physical machine vector dominant_share vector HS (hs_1, hs_2 ... hs_n), Value=J (TS)/J (HS) now and allocative decision is recorded.Wherein the computing method of the fair index of Jian Shi are as described below:

$J(x_1,x_2,...,x_n)=\frac{{\left({\mathrm{\&Sigma;}}_{i=1}^n{x}_i\right)}^2}{n*{\mathrm{\&Sigma;}}_{i=1}^n{x}_i^2}$

B9 scheduling of resource iterative estimation device calculates Value=J (TS)/J (HS), if the optimal value before being greater than, records the solution of this formula.

B10 scheduling of resource iterative combiner is chosen in the request sorted and is carried out association union operation from task strong continune is as calculated heavy, thus renewal merging merge_history skips to step B5, otherwise skips to step B11.

B11 algorithm terminates, and exports solution.

The parsing of associated task

The parsing of associated task is a step crucial in the pre-service link of algorithm.This step needs the incidence relation of the task to association to resolve.For dissimilar task, be different to the demand of resource.Need to carry out different associated tasks to the task of different association type to resolve.For uncorrelated task, do not need to ensure the locality of its data with communicate mutual, now the task location of the physical machine of platform all can as virtual machine open position candidate.And for there is the task DAG task of incidence relation, and the task of communication association then needs to parse this incidence relation when scheduling.The locality of data is considered when scheduling virtual machine.

This incidence relation is due to the combination of these three kinds of relations, and this negative assorted relational model can adopt the model of graph theory to be described and solve with modeling.The rule of modeling is as follows:

1) task is abstracted into the point in image, the correspondence between task is abstracted into the directed edge in graph theory.

2) for uncorrelated task, they are then described as an isolated node in the drawings.

3) for DAG task, if task B needs the end of wait task A to perform, relation so is in the drawings then one and points to the directed edge of B from A.

4) for inter-related communication task A and task B, their relation is in the drawings exactly that two directed edges are described, and is the directed edge that A points to that the directed edge of B and B point to A respectively.

By the relevant knowledge of graph theory, simultaneously in conjunction with the principle that scheduling of resource is distributed, take when the present invention carries out the distribution of task to observe following rule:

1) for uncorrelated task, Resources allocation can be carried out as the unit of an opposition.

2) for DAG task, by same DAG chain, should distribute using DAG chain as Resources allocation unit.

3) for inter-related communication task, model corresponding in graph theory is UNICOM's component of figure.During Resources allocation, in conjunction with the subset of UNICOM's component, should guarantee that the cost communicated can not become the bottleneck of system.

For the parsing of task, according to above-mentioned modeling rule, rule must be distributed and processes, in conjunction with graph-theoretical algorithm.The present invention takes the resolving of carrying out task according to flow process below.A2 resolves

Incidence relation between C1 extraction task, is described as the structure of a figure by the incidence relation of task.

C2, according to the relation of graph theory, calculates strong UNICOM component, marks each task and belongs to which UNICOM's component

C3 will in each UNICOM's component extraction out, and its assigned sequence sorts according to resource request number is ascending.The strong UNICOM component obtained is regarded as an overall task simultaneously.And by the model of figure, in conjunction with original relation, obtain a new task incidence relation.This task nexus being is a DAG task image.

DAG task image is carried out topological sorting by C4, obtains a topological sequences of new figure, and in each result of topological sequences, if having the new point that UNICOM's component point reduction causes, the sequence that this some step C3 obtains is replaced.

Based on the distribution of greedy algorithm

So in this patent, consider in conjunction with above-mentioned viewpoint.Following change is made at greedy algorithm resource allocator model.

Change resource mapping direction, in scheduling of resource in the past, always becomes resource pool by the Resource Abstract of platform, and is distributed as container by the request vector of user.In this patent.Resource pool is abstracted into, using the surplus resources of the physical machine on platform as the container distributed by the request of user during the direction of this mapping.During both present distribution, the resource of physical machine request dispatching task is on this virtual machine.

The resource request of multiple subtask is comprised, so can not be the process that linear summation is abstracted into request pond for scheduling of resource during resource request due to the operation of user.The request of user must be distributed as single entirety

For the request of the task of user, in order to ensure multiple resource distributional equity, retain the calculating of domainant_share and domainant_desire calculating business of changing to.

For the request dispatching vector of physical machine, in order to ensure the harmony of the load that multiple resource distributes, add the calculating of domainant_share and dominant_desire of computational physics machine request dispatching vector.

Its flow process in conjunction with concrete calculating is as follows: A3

D1 calculates dominant_share and dominant_desire of all task requests, and sorts according to its size.

D2 calculates the vector of dominant_share and dominant_desire of All hosts.Choose wherein first task, select the main frame that dominant_share and dominant_desire of the main frame that can distribute is minimum.

If D3 becomes the distribution of work, upgrade the vector of dominant_share and dominant_desire of All hosts.

If D4 has residue, main frame is also unallocated, continue above-mentioned steps D3, otherwise assigning process terminates.

The merging A5 of associated task

Under the environment of the platform of cooperated computing, the starting point of the merging of the resource request of task is in the sequence based on the parsing of incidence relation, for the merging of previous task A and rearmounted task B.For the task of association, according to the type of previous task and rearmounted task, following several situation can be had:

1) previous task and rearmounted task are communication association relations, so all must being met of two task resource requests during the merging of these two tasks to the request of resource.So the linear superposition of 2 task resource requests during the merging of resource in middle situation.

2) previous task and rearmounted task are DAG incidence relations, DAG relation during the resource request of so these two tasks.If these two tasks are not merged history.So compare the size cases of previous task and rearmounted task, retain larger.

3) if merged task when having in fruit previous task and rearmounted task, so merge according to following rule.What be set to merging is example later, and now the resource request of previous task can as a possibility candidate item of rearmounted task resource request.Therefore, when merging, need the history of the request traveling through rearmounted task merging, investigate the history of rearmounted task merging.

4) if there is the task requests sequence that every resource requirement is all less than preposition resource request in the resource requirement of rearmounted historic task.In rearmounted history, so ignore a rearmounted resource request wherein.The result result merged be request vector and the preposition resource requirement of difference rain of resource vector ignored of rearmounted task and.

5) if when the existence of preposition resource is less than a certain item task of rearmounted resource, ignore the resource requirement of this preposition task.

6) if not above-mentioned several situation, then merge according to 2 communication association tasks.

Experiment and test

Further, the present invention is from platform resource distributional equity, and testing and emulation has been carried out to dispatching platforms algorithm in these three aspects of the execution performance of platform resource utilization factor and platform.

Resourse Distribute fairness

Distributional equity is tested.30 tasks that test adopts stochastic generation to meet normal distribution carry out the experiment of virtual machine distribution to the cloud computing environment at above-mentioned environment.Adopt 30 identical tasks, adopt different dispatching algorithms to dispatch respectively.

Fig. 3 is the allocation result of resource fairness, as can be seen from the figure, the present invention the algorithm distributed based on multiple resource with based on single resource allocation allocation algorithm the fair standard of cpu and overall dominant_share be 0.99 than single resource algorithm have larger advantage, to be all in internal memory index slightly not enough.

Platform resource utilization factor

Further analysis, the resource utilization next for associated task carries out simulating and emulating, and the associated task of simulating generation 100 different carries out.The running status of record clustering simultaneously.

Fig. 4, Fig. 5 are the request of the cpu of parsing task respectively and resolve the memory request of task, and the result of emulation shows, the optimization method of the merging that the task of the dispatching method that the present invention takes uses can make reducing cpu and memory request of the task of 50%.As shown in Figure 6 simultaneously, adopt dispatching platform of the present invention in the number of tasks run of peak value than original dispatching algorithm many places 20%, further as shown in Figure 7, occur owing to decreasing idle waiting situation, the idle waiting task in the present invention only has 50% of original dispatching platform.Substantially increase the overall execution performance of platform, show from Fig. 6, Fig. 7, the time more original algorithm based on dispatching algorithm process of the present invention 100 tasks decreases about 25%, improves the handling property of integral platform.

Platform execution performance

The multinode program based on MPI of communication is adopted to test.20 tasks are carried out in test dispatches, and calculates the averaging time that it runs.Experimental result as shown in Figure 8, the more original task of dispatching algorithm of the present invention merged based on correlation rule is by the time delay optimization that calculates about 15%, and practice shows, if in the process calculated, the further increase of the traffic, optimization measure of the present invention will strengthen further.

Above content is only and technological thought of the present invention is described; protection scope of the present invention can not be limited with this; every technological thought proposed according to the present invention, any change that technical scheme basis is done, within the protection domain all falling into claims of the present invention.

Claims (9)

1. towards a schedule virtual resources system for associated task request, it is characterized in that: comprise resource management controller, resource request resolver and scheduling of resource iterator;

The resource situation of physics host of resource management controller for receiving resource monitoring and obtaining;

Resource request resolver is used for front end pretreatment process and resource request encapsulates;

Scheduling of resource iterator, for carrying out the iterative control process dispatched, produces scheduling of resource result.

2. the schedule virtual resources system towards associated task request according to claim 1, is characterized in that: described resource management controller specifically for:

1) dock with monitoring resource interface, Real-time Obtaining cloud computing cluster resource state status;

2) in conjunction with scheduling of resource iteration controller, extract the resource situation of each physical machine in platform from monitor data, resource situation comprises the corresponding state of cpu, internal memory, bandwidth and hard disk;

3) according to the resource situation of physical machine each in cloud computing cluster, when the controller request of scheduling iteration device, real-time calculate total resources in cluster and surplus resources total amount, calculate dominant_share and dominant_desire of each physical machine under current state simultaneously;

4) under concurrent environment, ensure the consistance of data, ensure the consistance of cluster state information; When the high concurrent operation of scheduling of resource iterator, provide the consistance to resource, atomicity and transactional.

3. the schedule virtual resources system towards associated task request according to claim 1, is characterized in that: the responsible following content that described resource request resolver is concrete:

1) receive the request json file of the task scheduling request user that framework is sent, extract the cpu that scheduling iteration device is concerned about, the resource request such as internal memory, simultaneously the resource request of further encapsulation user;

2) analytical framework task association request, runs strong UNICOM component, the graph-theoretical algorithms such as topological sorting, produces the task sequence with special marking;

3) provide the encapsulation operation of union operation, in the operation utilizing Union-find Sets, provide follow-up scheduling of resource iterator to carry out operating to carry out union operation, further scheduling of resource iterator can be inquired about and merge history generation scheduling of resource result.

4. the schedule virtual resources system towards associated task request according to claim 1, is characterized in that: described scheduling of resource iterator comprises scheduling of resource iteration controller, scheduling of resource iterative combiner, scheduling of resource iterative sequencing device and scheduling of resource iterative estimation device;

Scheduling of resource iteration controller, for carrying out the iteration control of dispatching, controls scheduling of resource scheduling iteration combiner and the collaborative work of scheduling of resource sorting unit, judges stopping criterion for iteration; And in the iterative process of iteration, resource request is distributed, carry out the calculating of the resource of actual distribution;

Scheduling of resource iterative combiner is responsible for the operation carrying out merging request according to the design of CBDRF algorithm, according to the algorithm of CBDRF design, adopts the method for Union-find Sets, carries out union operation according to the markd task sequence of band that resource request resolver sends;

Scheduling of resource iterative sequencing device is used for auxiliary resources scheduling iteration controller in the process of carrying out Resourse Distribute and carries out Resourse Distribute;

Scheduling of resource iterative estimation device, for assessment of scheduling result, records current assessment result; The scheduling intermediate result simultaneously produced with previous iteration as calculated compares, if current assessed value is better than previous scheduling result, recording dispatching result is current results, otherwise give up current result, notice scheduling of resource iteration controller carries out next round iterator.

5., based on the schedule virtual resources towards associated task request and the distribution method of system described in claim 1-4 any one, it is characterized in that, comprise the following steps:

A1 resource management controller obtains the resource status of cloud computing platform, carries out corresponding pretreatment operation;

A2 resource request resolver connects resolves task status according to the correlation model of graph theory, generates corresponding task sequence;

A3 scheduling of resource iterator carries out the assigning process based on max_min operation, generates iteration scheduling result;

A4 scheduling of resource iterative estimation device assessment result, preserves the optimal value calculated;

A5 carries out the union operation of associated task according to the model of graph theory, merges corresponding task requests;

Can A6 scheduling of resource iteration controller judge distribute, if can go to step 3), otherwise Output rusults.

6. the schedule virtual resources towards associated task request according to claim 5 and distribution method, is characterized in that, in described steps A 1 to A4, the dispatching distribution strategy of scheduling of resource iteration control implement body carries out in accordance with the following steps:

B1 resource resolver resolves user ask, be each request initialization request example, each example be five-tuple wherein:

Five-tuple comprises cpu, mem, related_request, merge_history, scheduled_host; Cpu represents the cpu check figure of the virtual machine of task requests; Mem represents the virutal machine memory size of task requests, in units of GB; Related_request represents associated task request set; Merge_history represents merging history, and scheduling iteration device resolves final scheduling result in conjunction with scheduled_host from merging history; Scheduled_host represents the scheduling result of request, is resolved the scheduling result of final request by merge_history;

The initial value of above-mentioned cpu, mem, related_request obtains from the request of user, merge_history and scheduled_host just initial set is empty set;

B2 is abstracted into a little by there being mutually the task of interchange, and network service relation between them is abstracted into limit, by related_request, request analysis is abstracted into a graph of a relation; Run the strong UNICOM component of graph-theoretical algorithm calculation task graph of a relation by resource resolver, be marked in request example;

B3request example sends resolution data to scheduling of resource iterator, and simultaneously scheduling of resource iterator sends request to resource management controller, with obtain cloud computing platform real time resources state;

B4 resource management controller is that each physical machine creates host example, and each example is four-tuple, wherein:

Four-tuple comprises cpu_total, cpu_used, mem_total, mem_used; Cpu represents that single physical machine represents cpu check figure respectively; Cpu_used represents the cpu check figure that single physical machine has used; Mem_total represents single physical machine internal memory sum, in units of GB; Mem_uesd represents that single physical has used internal memory sum, in units of GB;

Above-mentioned numerical value all obtains these numerical value by the monitor-interface of cloud computing platform;

B5 using the request of user as the resource pool that will distribute, by abstract for these resource request total resources for distributing to physical machine when multiple resource request arrives time, the capacity of physical machine is calculated the dominant_share of each physical machine host as the request vector of reality, in resource vector, maximum resource occupation rate is called the dominant_share of physical machine simultaneously; Maximal value in surplus resources vector becomes the dominant_desire of this physical machine; Its computing formula is as follows:

The total number vector of cloud computing resources:

R＝<r ₁， … r _m>

Single physical machine i total resources:

HT _i＝<ht _i，1， … ht _i，m>

Single physical machine i is Resources allocation total amount:

HU _i＝<hu _i，1， … hu _i，m>

The dominant_share computing method of single physical machine i:

${\mathrm{HS}}_i={\max }_{j=1}^m\{{\mathrm{hu}}_{i,j}/r_j\}$

The dominant_desire computing method of single physical machine i:

${\mathrm{HD}}_i={\max }_{j=1}^m\{{\mathrm{ht}}_{i,j}/{\mathrm{hu}}_{i,j}\}$

B6 resource iterative combiner calculates dominant_share and dominant_desire of request, and according to the ascending sequence of the size of dominant_share, identical dominant_share then sorts according to the size of dominant_desire is ascending; Wherein dominant_share and dominant_desire of request is as described below:

Task requests total number resource vector:

T＝<t ₁， … t _m>

Task requests single physical machine i request resource total amount:

TR _i＝<tr _i，1， … tr _i，m>

The dominant_share computing method of task requests physical machine i:

${\mathrm{TS}}_i={\max }_{j=1}^m\{{\mathrm{tr}}_{i,j}/t_j\}$

The dominant_desire computing method of task requests single physical machine i:

${\mathrm{TD}}_i={\max }_{j=1}^m\left\{{\mathrm{tr}}_{i,j}\right\}$

B7 scheduling of resource iterative sequencing device and scheduling of resource iterator computing greedy algorithm;

B8 scheduling of resource iterative estimation device utilizes the fair index of Jain ' s fairness index: Jian Shi, the dominant_share vector T S (ts_1 of computational resource request request, ts_2, ts_k), cloud computing platform physical machine vector dominant_share vector HS (hs_1, hs_2 ... hs_n), Value=J (TS)/J (HS) now and allocative decision is recorded; Wherein the computing method of the fair index of Jian Shi are as described below:

$J(x_1,x_2,...,x_n)=\frac{{\left({\mathrm{\&Sigma;}}_{i=1}^n{x}_i\right)}^2}{n*{\mathrm{\&Sigma;}}_{i=1}^n{x}_i^2}$

B9 scheduling of resource iterative estimation device calculates Value=J (TS)/J (HS), if the optimal value before being greater than, records the solution of this formula;

B10 scheduling of resource iterative combiner is chosen in the request sorted and is carried out association union operation from task strong continune is as calculated heavy, thus renewal merging merge_history skips to step B5, otherwise skips to step B11;

B11 algorithm terminates, and exports solution.

7. the schedule virtual resources towards associated task request according to claim 5 or 6 and distribution method, is characterized in that, in described steps A 2, the concrete grammar of resolving task is:

Incidence relation between C1 extraction task, is described as the structure of a figure by the incidence relation of task;

C2, according to the relation of graph theory, calculates strong UNICOM component, marks each task and belongs to which UNICOM's component;

By each strong UNICOM component extraction out, its assigned sequence sorts according to resource request number is ascending C3; The strong UNICOM component obtained is regarded as an overall task, and by the model of figure simultaneously, in conjunction with original relation, obtain a new task incidence relation; By original complex task figure boil down to DAG task image;

DAG task image is carried out topological sorting by C4, obtains a topological sequences of new figure, and in each result of topological sequences, if having the new point that UNICOM's component point reduction causes, the sequence that this some step C3 obtains is replaced.

8. the schedule virtual resources towards associated task request according to claim 5 or 6 and distribution method, is characterized in that, in described steps A 3, the concrete calculation process based on the assigning process of max_min operation carries out in accordance with the following steps:

D1 calculates dominant_share and dominant_desire of all task requests, and sorts according to its size;

D2 calculates the vector of dominant_share and dominant_desire of All hosts; Choose wherein first task, select the main frame that dominant_share and dominant_desire of the main frame that can distribute is minimum;

If D3 becomes the distribution of work, upgrade the vector of dominant_share and dominant_desire of All hosts;

If D4 has residue, main frame is also unallocated, continue above-mentioned steps D3, otherwise assigning process terminates.

9. the schedule virtual resources towards associated task request according to claim 5 and distribution method, is characterized in that, in described steps A 5, associated task union operation carries out as follows according to situation:

1) if previous task and rearmounted task are communication association relations, so all must being met of two task resource requests during the merging of these two tasks to the request of resource, in this situation situation, the amalgamation result of resource is the linear superposition of 2 task resource requests;

2) if previous task and rearmounted task are DAG incidence relations, so the resource request of these two tasks is DAG relations; If these two tasks are not merged history, so compare the size cases of previous task and rearmounted task, retain larger;

3) if merged task when having in fruit previous task and rearmounted task, so merge according to following rule: be set to later merging for example, now the resource request of previous task may candidate item as one of rearmounted task resource request; Therefore, when merging, need the history of the request traveling through rearmounted task merging, investigate the history of rearmounted task merging;

4) if there is the task requests sequence that every resource requirement is all less than preposition resource request in the resource requirement of rearmounted historic task, in rearmounted history, so ignore a rearmounted resource request wherein, the result of merging be the request vector of rearmounted task with the difference of the resource vector ignored and preposition resource requirement with;

5) if when the existence of preposition resource is less than a certain item task of rearmounted resource, ignore the resource requirement of this preposition task;

6) if not above-mentioned several situation, then merge according to 2 communication association tasks.