CN103257896A - Max-D job scheduling method under cloud environment - Google Patents
Max-D job scheduling method under cloud environment Download PDFInfo
- Publication number
- CN103257896A CN103257896A CN2013100383292A CN201310038329A CN103257896A CN 103257896 A CN103257896 A CN 103257896A CN 2013100383292 A CN2013100383292 A CN 2013100383292A CN 201310038329 A CN201310038329 A CN 201310038329A CN 103257896 A CN103257896 A CN 103257896A
- Authority
- CN
- China
- Prior art keywords
- resource
- time
- cloud environment
- max
- deadline
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention discloses a Max-D job scheduling method under the cloud environment. The method includes estimating running time of each task on resources, using a Max-D algorithm for selecting an optimum resource for each task to execute, and when actual running efficiency is higher than estimated efficiency, matching the tasks with the resources. By the job scheduling method, load balance of the resources under the cloud environment can be guaranteed effectively, average job running time can be shortened, and system throughput is increased.
Description
Technical field
The present invention relates to the cloud environment job scheduling method, the Max-D job scheduling method under especially a kind of cloud environment.
Technical background
In recent years, the cloud computing mode development is rapid, and framework and the method for operation of IT industry are also changed thereupon.Cloud computing reduces the requirement of computing power, and the management mode of simultaneous altitude robotization seldom needs manual intervention, has saved enterprise procurement and artificial cost in a large number.This also makes high-performance computer, high-end storer, high-end server market be tied up by the cluster of low side devices gradually; Traditional data center is replaced by cloud computing center with low cost; A large amount of software application are released at the cloud platform with service manner, even much use and play and can move in " cloud ".
Cloud computing workload to be processed and data volume be huge, system is almost constantly all in operation and the data of handling magnanimity, therefore how the cloud resource is reasonably distributed, operation is dispatched efficiently, make it possible to satisfy user's user demand, make the processing time of the operation that the user submits to short, execution cost is less, the state that the load of simultaneity factor maintains a relative equilibrium is emphasis and the difficult point in the cloud computing.(Quality of Service, demand QOS) is so study the significant of job scheduling method under the cloud environment just because of needing farthest to satisfy QoS of customer under the cloud environment.Inappropriate job scheduling strategy can cause the waste of cloud resource, increases execution time and the cost of user job, when serious even make system congestion can't continue to provide service; And suitable job scheduling can reduce the wasting of resources under the prerequisite of the demand that satisfies the user as far as possible, reaches the expection of user and cloud service provider.Therefore, the job scheduling method under a kind of cloud environment that can satisfy user and enterprise demand of research is necessary.
A. Vouk has proposed the Min-Min job scheduling method in " Cloud Computing-Issues, Research and Implementations ".The Min-Min dispatching method estimates that minimum deadline of operation that each does not dispatch obtains minimum deadline set, and the minimum deadline with All Jobs compares then, chooses in the set minimum operation branch of deadline and tasks suitable computing node.The Min-Min method can make that the deadline of single homework is less, but can produce the unbalanced of load, and the flat near deadline of operation is longer.
Summary of the invention
The purpose of this invention is to provide the job scheduling method under a kind of cloud environment, can keep load balancing when making under the cloud environment computing node processing operation, and improve the average deadline of operation.
The technical scheme that realizes the object of the invention is:
Max-D job scheduling method under a kind of cloud environment, step is as follows:
The first step: determine the set of all computational resources in the cloud environment and idling-resource.
Second step: submit to priority to be ranked into formation to operation to be allocated by operation, the new operation of submitting to is added into this formation afterbody.
The 3rd step: the operation after the ordering is dispatched, adopt the Max-D method to select adequate resources to carry out.
For the Max-D method in the 3rd step, its step is as follows:
Step 3.1: to all operations to be allocated, the average estimation working time of computational tasks on all computational resources;
Step 3.2: average estimation working time of calculating each operation with and difference Di between working time minimum on the computational resource of single free time, and record this computational resource;
Step 3.3: in All Jobs, find the operation of difference Di maximum, and this Di is designated as D;
Step 3.4: if
, then assign operation and handle for the resource of record, simultaneously this resource is removed from the idling-resource set; If
, then redefine resource and the idling-resource set of distribution, join in the idling-resource set finishing its resource of distributing operation, return step 3.1 then.
Step 3.5: repeating step 3.2 has distributed operation to step 3.4 up to the resource of applying for operations for all.
The average estimated time to completion method of computational resource is as follows in the step 3.1:
Suppose that cloud environment is by n unallocated operation
With m resource
Form, each resource can only be handled an operation simultaneously; Resource number idle in the resource is k, is designated as
, k＜m wherein; The estimation working time of operation ti on resource rj is TCirj, and then the average operating time of operation ti on all resources is
The deadline of operation ti on resource rj is residue deadline and the deadline sum of operation ti on rj of the operation just carried out at rj.
Suppose that in cloud environment for same class operation, the speed that resource is handled is directly proportional with the data volume of its processing.Operation i is just in residue deadline and operation i execution time sum on resource r of running job on the resource r at the Estimated Time Of Completion on the resource r:
Wherein,
Expression is the required deadline of resource rj processing operation ti,
Represent the prediction deadline of previous operation on resource rj;
Be the ratio of this operation required time of operation and run unit operation required time;
Represent previous operation actual run time on rj,
The completed percentage of representing previous operation, if resource rj is idling-resource, i.e. previous operation is complete, then
, above-mentioned formula can be reduced to
（2）
By the estimation execution time of previous operation on this resource
And actual execution time
, use formula (1) to estimate the execution time of operation on certain resource of not dispatched.Yet, in the stage that system has just started, also do not carry out operation on each resource, the execution time of resource can't be estimated by the implementation status of previous operation.When therefore just starting in system, for all resources, order
（3）
The resource that pending like this operation meeting is at first selected not carry out operation is carried out, and after resource executes first operation, has just obtained the actual execution time of operation
, order
Equal
, then estimate according to formula (1) working time of operation afterwards.
The method of calculated difference D is as follows in the step 3.2:
Operation ti is designated as the minimum working time on the node of all unallocated work
, note satisfied
Unallocated operation rj ', and the note
, then according to formula
, obtain the difference Di of operation i.
The present invention compared with prior art, its remarkable advantage: 1, compared to the traditional scheduler method, job scheduling of the present invention only can be assigned to operation on the idle resource, has guaranteed the equilibrium of load under the cloud environment, the part resource overload can not occur and the situation of other resource free time;
2, compared to the traditional scheduler method, the present invention is that only resource is selected in operation by the Max-D method, has reduced the average deadline of operation, has improved the throughput of system.
Description of drawings
Accompanying drawing is the process flow diagram of Max-D method of the present invention.
Embodiment
The present invention will be further described below in conjunction with accompanying drawing.
Suppose that cloud environment is by n unallocated operation
With m resource
Form, each resource can only be handled an operation simultaneously; Resource number idle in the resource is k, is designated as
, k＜m wherein; The estimation working time of operation ti on resource rj is TCirj, and then the average operating time of operation ti on all resources is
Operation ti is designated as the minimum working time on the node of all unallocated work
, note satisfied
Unallocated operation rj ', and the note
When non-NULL is gathered in the operation of needs scheduling, carry out following operation:
Step 1: All Jobs among the operation set T is calculated respectively
Step 2: each operation ti is found
, and calculate
Step 3: find operation ti, make
, if there are a plurality of operations to satisfy condition, the select progressively ti that arrives according to these operations then;
Step 4: if
, then assign operation ti and handle for resource BRi, resource BRi is gathered the R ' from idling-resource remove simultaneously; If
, resource and the idling-resource set of then reappraising and distributing join in the idling-resource set finishing its resource of distributing operation, return step (1) then.
Step 5: repeating step 2 has distributed operation to step 4 up to the resource of applying for operations for all.
The deadline of operation ti on resource rj is residue deadline and the deadline sum of operation ti on rj of the operation just carried out at rj.
This document assumes that is for same class operation, and the speed that resource is handled is directly proportional with the data volume of its processing.Operation i is just in residue deadline and operation i execution time sum on resource r of running job on the resource r at the Estimated Time Of Completion on the resource r:
Wherein,
Expression is distributed to resource rj with operation ti and is handled the required deadline,
Represent the prediction deadline of previous operation on resource rj;
Be the ratio of this operation required time of operation and run unit operation required time;
Represent previous operation actual run time on rj,
The completed percentage of representing previous operation, if resource rj is idling-resource, i.e. previous operation is complete, then
, above-mentioned formula can be reduced to
According to formula, Tiao Du operation can be by the estimation execution time of previous operation on this resource in the execution time on certain resource
And actual execution time
Estimate.Yet, in the stage that system has just started, also do not carry out operation on each resource, the execution time of resource can't be estimated by the implementation status of previous operation.When therefore just starting in system, for all resources, order
Pending like this operation meeting at first selects not carry out the resource of operation, after resource executes first operation, has just obtained the actual execution time of operation
, order
Equal
, then can estimate according to formula (1) working time of operation afterwards.
Claims (4)
1. the Max-D job scheduling method under the cloud environment is characterized in that step is as follows:
The first step: determine the set of all computational resources in the cloud environment and idling-resource;
Second step: submit to priority to be ranked into formation to operation to be allocated by operation, the new operation of submitting to is added into this formation afterbody;
The 3rd step: the operation after the ordering is dispatched, adopt the Max-D method to select adequate resources to carry out.
2. the Max-D job scheduling method under the cloud environment according to claim 1 is characterized in that, the Max-D method in described the 3rd step, and its step is as follows:
Step 3.1: to all operations to be allocated, the average estimation working time of computational tasks on all computational resources;
Step 3.2: average estimation working time of calculating each operation with and difference Di between working time minimum on the computational resource of single free time, and record this computational resource;
Step 3.3: in All Jobs, find the operation of difference Di maximum, and this Di is designated as D;
Step 3.4: if
, then assign operation and handle for the resource of record, simultaneously this resource is removed from the idling-resource set; If
, then redefine resource and the idling-resource set of distribution, join in the idling-resource set finishing its resource of distributing operation, return step 3.1 then;
Step 3.5: repeating step 3.2 has distributed operation to step 3.4 up to the resource of applying for operations for all.
3. the Max-D job scheduling method under the cloud environment according to claim 1 is characterized in that, the average estimated time to completion method of computational resource is as follows in the described step 3.1:
Suppose that cloud environment is by n unallocated operation
With m resource
Form, each resource can only be handled an operation simultaneously; Resource number idle in the resource is k, is designated as
, k＜m wherein; The estimation working time of operation ti on resource rj is TCirj, and then the average operating time of operation ti on all resources is
The deadline of operation ti on resource rj is residue deadline and the deadline sum of operation ti on rj of the operation just carried out at rj;
Suppose in cloud environment, for same class operation, the speed that resource is handled is directly proportional with the data volume of its processing, and operation i is just in residue deadline and operation i execution time sum on resource r of running job on the resource r at the Estimated Time Of Completion on the resource r:
Wherein,
Expression is the required deadline of resource rj processing operation ti,
Represent the prediction deadline of previous operation on resource rj;
Be the ratio of this operation required time of operation and run unit operation required time;
Represent previous operation actual run time on rj,
The completed percentage of representing previous operation, if resource rj is idling-resource, i.e. previous operation is complete, then
, above-mentioned formula can be reduced to
By the estimation execution time of previous operation on this resource
And actual execution time
, use formula (1) to estimate the execution time of operation on certain resource of not dispatched;
When system just starts, for all resources, order
The resource that pending operation meeting is at first selected not carry out operation is carried out, and after resource executes first operation, has just obtained the actual execution time of operation
, order
Equal
, then estimate according to formula (1) working time of operation afterwards.
4. the Max-D job scheduling method under the cloud environment according to claim 1 is characterized in that, the method for calculated difference D is as follows in the step 3.2:
Operation ti is designated as the minimum working time on the node of all unallocated work
, note satisfied
Unallocated operation rj ', and the note
, then according to formula
, obtain the difference Di of operation i.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310038329.2A CN103257896B (en) | 2013-01-31 | 2013-01-31 | A kind of Max-D job scheduling method under cloud environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310038329.2A CN103257896B (en) | 2013-01-31 | 2013-01-31 | A kind of Max-D job scheduling method under cloud environment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103257896A true CN103257896A (en) | 2013-08-21 |
| CN103257896B CN103257896B (en) | 2016-09-28 |
Family
ID=48961831
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310038329.2A Expired - Fee Related CN103257896B (en) | 2013-01-31 | 2013-01-31 | A kind of Max-D job scheduling method under cloud environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103257896B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103530182A (en) * | 2013-10-22 | 2014-01-22 | 海南大学 | Working scheduling method and device |
| CN104951368A (en) * | 2014-03-28 | 2015-09-30 | 中国电信股份有限公司 | Dynamic allocation device and method of resources |
| CN105446979A (en) * | 2014-06-27 | 2016-03-30 | 华为技术有限公司 | Data mining method and node |
| CN106790636A (en) * | 2017-01-09 | 2017-05-31 | 上海承蓝科技股份有限公司 | A kind of equally loaded system and method for cloud computing server cluster |
| CN108270833A (en) * | 2016-12-31 | 2018-07-10 | 中国移动通信集团安徽有限公司 | Render automatic scheduling method, the apparatus and system of cloud resource |
| CN108509256A (en) * | 2017-02-28 | 2018-09-07 | 华为技术有限公司 | Method, equipment and the running equipment of management and running equipment |
| CN110445939A (en) * | 2019-08-08 | 2019-11-12 | 中国联合网络通信集团有限公司 | The prediction technique and device of capacity resource |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1430366A (en) * | 2001-12-29 | 2003-07-16 | 深圳市中兴通讯股份有限公司 | Method and device of continuous channet unit dispatching used in CDMA system |
| US20110202657A1 (en) * | 2010-02-12 | 2011-08-18 | Elitegroup Computer Systems Co., Ltd. | Method for scheduling cloud-computing resource and system applying the same |
-
2013
- 2013-01-31 CN CN201310038329.2A patent/CN103257896B/en not_active Expired - Fee Related
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1430366A (en) * | 2001-12-29 | 2003-07-16 | 深圳市中兴通讯股份有限公司 | Method and device of continuous channet unit dispatching used in CDMA system |
| US20110202657A1 (en) * | 2010-02-12 | 2011-08-18 | Elitegroup Computer Systems Co., Ltd. | Method for scheduling cloud-computing resource and system applying the same |
Non-Patent Citations (2)
| Title |
|---|
| 牛川川: ""计算网格中任务调度算法和策略的研究"", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 葛新: ""基于云计算集群扩展中的调度问题研究"", 《中国优秀硕士学位论文全文数据库 经济与管理科学辑》 * |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103530182A (en) * | 2013-10-22 | 2014-01-22 | 海南大学 | Working scheduling method and device |
| CN104951368B (en) * | 2014-03-28 | 2019-02-22 | 中国电信股份有限公司 | Resource dynamic allocation device and method |
| CN104951368A (en) * | 2014-03-28 | 2015-09-30 | 中国电信股份有限公司 | Dynamic allocation device and method of resources |
| CN105446979A (en) * | 2014-06-27 | 2016-03-30 | 华为技术有限公司 | Data mining method and node |
| US10102038B2 (en) | 2014-06-27 | 2018-10-16 | Huawei Technologies Co., Ltd. | Data mining method and node |
| CN105446979B (en) * | 2014-06-27 | 2019-02-01 | 华为技术有限公司 | Data digging method and node |
| CN108270833A (en) * | 2016-12-31 | 2018-07-10 | 中国移动通信集团安徽有限公司 | Render automatic scheduling method, the apparatus and system of cloud resource |
| CN108270833B (en) * | 2016-12-31 | 2021-07-16 | 中国移动通信集团安徽有限公司 | Automatic scheduling method, device and system for rendering cloud resources |
| CN106790636A (en) * | 2017-01-09 | 2017-05-31 | 上海承蓝科技股份有限公司 | A kind of equally loaded system and method for cloud computing server cluster |
| CN108509256B (en) * | 2017-02-28 | 2021-01-15 | 华为技术有限公司 | Method and device for scheduling running device and running device |
| CN108509256A (en) * | 2017-02-28 | 2018-09-07 | 华为技术有限公司 | Method, equipment and the running equipment of management and running equipment |
| CN110445939A (en) * | 2019-08-08 | 2019-11-12 | 中国联合网络通信集团有限公司 | The prediction technique and device of capacity resource |
| CN110445939B (en) * | 2019-08-08 | 2021-03-30 | 中国联合网络通信集团有限公司 | Capacity resource prediction method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103257896B (en) | 2016-09-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103257896A (en) | Max-D job scheduling method under cloud environment | |
| CN110297699B (en) | Scheduling method, scheduler, storage medium and system | |
| CN103870340B (en) | Data processing method, control node and stream calculation system in stream calculation system | |
| US9471390B2 (en) | Scheduling mapreduce jobs in a cluster of dynamically available servers | |
| CN109582448B (en) | Criticality and timeliness oriented edge calculation task scheduling method | |
| CN105260005B (en) | Cloud workflow schedule optimization method towards energy consumption | |
| CN103401939A (en) | Load balancing method adopting mixing scheduling strategy | |
| CN102902592B (en) | A kind of subregion schedule management method of PC cluster resource | |
| CN105373429A (en) | Task scheduling method, device and system | |
| CN103530182A (en) | Working scheduling method and device | |
| CN102243598A (en) | Task scheduling method and system in distributed data warehouse | |
| CN103338228A (en) | Cloud calculating load balancing scheduling algorithm based on double-weighted least-connection algorithm | |
| CN109784663B (en) | Workflow scheduling method and device | |
| CN110308967A (en) | A kind of workflow cost based on mixed cloud-delay optimization method for allocating tasks | |
| CN104023042A (en) | Cloud platform resource scheduling method | |
| CN103455375A (en) | Load-monitoring-based hybrid scheduling method under Hadoop cloud platform | |
| CN103997515B (en) | Center system of selection and its application are calculated in a kind of distributed cloud | |
| CN107292419A (en) | The Cost Optimization strategy that dynamic Multi-workflow scheduling is performed in a kind of mixing cloud environment | |
| CN115134371A (en) | Scheduling method, system, equipment and medium containing edge network computing resources | |
| CN104598311A (en) | Method and device for real-time operation fair scheduling for Hadoop | |
| CN110084507B (en) | Scientific workflow scheduling optimization method based on hierarchical perception in cloud computing environment | |
| Shu-Jun et al. | Optimization and research of hadoop platform based on fifo scheduler | |
| JP2021504777A5 (en) | ||
| CN112306696A (en) | Energy-saving and efficient method and system for unloading edge computing tasks | |
| CN104915250B (en) | It is a kind of to realize the method for making MapReduce data localization in the industry |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20170713 Address after: High tech Zone Nanjing city Jiangsu province 210000 Liufang Road No. 8 Building 7 layer Patentee after: Nanjing China Network Technology Co., Ltd. Address before: 222000 No. 2 Chenguang Road, Sinpo District, Jiangsu, Lianyungang Patentee before: Lianyungang Research Institute of Nanjing University of Science and Technology |
|
| TR01 | Transfer of patent right | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20160928 Termination date: 20200131 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |