CN101604261B - Task scheduling method for supercomputer - Google Patents
Task scheduling method for supercomputer Download PDFInfo
- Publication number
- CN101604261B CN101604261B CN 200910108710 CN200910108710A CN101604261B CN 101604261 B CN101604261 B CN 101604261B CN 200910108710 CN200910108710 CN 200910108710 CN 200910108710 A CN200910108710 A CN 200910108710A CN 101604261 B CN101604261 B CN 101604261B
- Authority
- CN
- China
- Prior art keywords
- task
- computing node
- node
- computing
- scheduling
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Supply And Distribution Of Alternating Current (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention provides a task scheduling method for a supercomputer, which comprises the following steps: A, acquiring the residual computing power of computing nodes and distances between the computing nodes and a node where data of a task to be processed is positioned; B, sorting the computing nodes from near to far according to the distances between the computing nodes and the data of the task to be processed; and C, scheduling the task to the computing nodes according to a sorting result. The task scheduling method for the supercomputer can improve operating efficiency.
Description
Technical field
The present invention relates to field of computer technology, more particularly, relate to a kind of method for scheduling task of supercomputer.
Background technology
That supercomputer is meant is that a plurality of computing nodes combine, can the parallel computing machine that carries out large-scale calculations or data processing, be also referred to as parallel computer.Supercomputer is because its powerful calculation process ability now is applied to fields such as industry, scientific research, science more and more.At present, the method for scheduling task in the supercomputer is dispatched to be calculated as the center mostly.Particularly, be the residue computing power of obtaining each computing node, search and obtain satisfying the node of task, and task scheduling is arrived this node the Capability Requirement of node.
Result according to task scheduling tends to cause great amount of data transmission, particularly for data-intensive application and data center, for example based on the data center of search engine business, the cost plenty of time is carried out the operational efficiency that data transmission will reduce supercomputer, thereby has also reduced processing speed of data.
Summary of the invention
Based on this, be necessary to provide a kind of method for scheduling task that can improve the supercomputer of operational efficiency.
The method for scheduling task of described supercomputer comprises: A. obtains the residue computing power of computing node and the distance between described computing node and the pending data of the task place node; B. extremely far described computing node is sorted by near by the distance between described computing node and the pending data of the task place node; C. according among the step B computing node being sorted the ranking results scheduler task that obtains to computing node; Described step C comprises:
According to described ranking results by near to searching computing node far successively, when the surplus capacity of first computing node does not satisfy task to the Capability Requirement of node, the volume of transmitted data that more following scheduling mode produces: after the parallel section of the treatable task of first computing node is dispatched to first computing node, with the part task immigration that moves on described first computing node on the nearest node of first computing node, satisfy task scheduling remaining parallel section behind first computing node until the residue computing power of described first computing node Capability Requirement of node and is dispatched to the volume of transmitted data that described first computing node is produced with the remaining parallel section of task; The parallel section of the treatable described task of residue computing power of described first computing node is dispatched to first computing node, extremely far find second computing node according to described ranking results by near, and the remaining parallel section of described task is dispatched to the volume of transmitted data that second computing node is produced; Scheduling mode is made as the less scheduling mode of volume of transmitted data of generation; Described step C selects data quantity transmitted is little and data rate is the fastest scheduling mode that the remaining parallel section of task that the parallel section of the treatable task of first computing node is dispatched to behind first computing node is dispatched to respective nodes.
Preferably, described step C comprises: according to described ranking results by near to searching computing node far successively, when first computing node satisfies task to the Capability Requirement of node, with task scheduling to described computing node.
Preferably, described step C can comprise: according to described ranking results by near to searching computing node far successively, when the residue computing power of first computing node does not satisfy described task to the Capability Requirement of node, moving on described first computing node of task is moved, residue computing power until described first computing node satisfies the Capability Requirement of task to node, and task scheduling is arrived described first computing node.
Described step C also can comprise: according to described ranking results by near to searching computing node far successively, when the residue computing power of first computing node does not satisfy described task to the Capability Requirement of node but greater than the minimum thread of described task, the parallel section of the treatable described task of residue computing power of described first computing node is dispatched to first computing node.
Further preferably, described parallel section with task also comprises after being dispatched to first computing node: moving on described first computing node of task is moved, residue computing power until described first computing node can be handled the remaining parallel section of described task, and the remaining parallel section of described task is dispatched to described first computing node.
Further preferably, the described step that moving on first computing node of task is moved is: with the task immigration that moves on first computing node to the 3rd nearest computing node of described first computing node of distance.
Preferably, described parallel section with task is dispatched to and also comprises after first computing node: by described ranking results by near to searching second computing node far successively, and described task scheduling remaining parallel section behind first computing node is dispatched to described second computing node.
The method for scheduling task of above-mentioned supercomputer, by obtaining the distance between the node of the pending data of computing node and task place, task scheduling is moved to the near as far as possible computing node of the pending data of affair place node of leaving one's post, can effectively reduce the migration of data, thereby improve the operational efficiency of supercomputer.
Description of drawings
Fig. 1 is the process flow diagram of the method for scheduling task of supercomputer among the embodiment;
Fig. 2 is according to the method flow diagram of ranking results scheduler task to computing node among the embodiment;
Fig. 3 is according to the method flow diagram of ranking results scheduler task to computing node among another embodiment.
Embodiment
Fig. 1 shows the method for scheduling task flow process of supercomputer among the embodiment, and detailed process is as follows:
In step S101, obtain the residue computing power of computing node and the distance between described computing node and the pending data of the task place node.
In step S102, extremely far described computing node is sorted by near by the distance between described computing node and the pending data of the task place node.
In step S103, arrive computing node according to described ranking results scheduler task.
The residue computing power of aforementioned calculation node, be meant that each computing node in the supercomputer can be for the remaining ability of domination, information such as for example remaining computing power (can be flops etc.), remaining storage capacity (can be memory size etc.), the remaining network bandwidth byte number of per second transmission (can be etc.) and temperature.Then can judge this node operation task of whether having the ability according to the residue computing power of computing node.
Distance between the node of the pending data of aforementioned calculation node and task place is meant the distance of data transmission between the node of the pending data of computing node and task place, and when distance is far away more, then data transmission is slow more, and distance is near more, and then data transmission is fast more.In one embodiment, the required deposit data to be processed of task T (for example a certain data search task) is on certain node (node N0), and other computing node in the supercomputer and the distance between the node N0 to transmit data between N1, N2...... node N1 and the node N0 the fastest to far being respectively by near, take second place successively.
Fig. 2 shows the method flow that arrives computing node among the embodiment according to the ranking results scheduler task, and detailed process is as follows:
In step S201, according to the ranking results of computing node by near to searching computing node far successively.As mentioned above, suppose to leave one's post be engaged in the distance of node N0 at pending data place by near to far being respectively node N1, N2......
In step S202, whether the residue computing power of decision node Ni (i=1,2......) satisfies the Capability Requirement of task to node, if, then enter step S203, otherwise, search next node, return step S202.In one embodiment, according to ranking results by near to far searching computing node Ni, judge at first whether the residue computing power from the nearest node N1 of node N0 satisfies the Capability Requirement of task to node, if, then can directly task scheduling be arrived node 1, otherwise, carry out i=i+1, promptly search next computing node N2, whether the residue computing power of decision node N2 satisfies the Capability Requirement of task to node, by that analogy, satisfy the Capability Requirement of task, then task scheduling is arrived this node node up to finding the residue computing power.The task here to the Capability Requirement of node can be: move this mission requirements memory size, move this mission requirements flops, move this mission requirements bandwidth, move in the parameters such as node temperature restriction of this task one or more.
In step S203, task scheduling is satisfied on the computing node of task to the Capability Requirement of node to the residue computing power.
Fig. 3 is the method flow that arrives computing node among another embodiment according to the ranking results scheduler task, and detailed process is as follows:
In step S301, according to the ranking results of computing node by near to searching computing node far successively.As mentioned above, suppose to leave one's post be engaged in the distance of node N0 at pending data place by near to far being respectively node N1, N2.......
In step S302, whether the residue computing power of decision node Ni (i=1,2......) satisfies the Capability Requirement of task to node, if, then enter step S303, otherwise, step S304 entered.In one embodiment, judge at first whether the residue computing power from the nearest node N1 of node N0 satisfies the Capability Requirement of task to node.
In step S303, task scheduling is arrived node Ni.In one embodiment, when the residue computing power of the node Ni that finds satisfies the Capability Requirement of task to node, then directly with task scheduling to this node Ni.For example, when the residue computing power that finds from the nearest node N1 of node N0 can satisfy the Capability Requirement of task to node, then task scheduling is arrived node N1.Because the distance of node N1 and the pending data of task place node N0 is the shortest, data transmission is also the fastest, has therefore improved the operational efficiency of supercomputer.
In step S304, whether the residue computing power of decision node Ni greater than the minimum thread of task, if, then enter step S305, otherwise, search next node, return step S302.In one embodiment, needing the task of scheduling is concurrent program, comprise a plurality of parallel sections, when the residue computing power of node Ni can not satisfy the Capability Requirement of task to node, and during less than the minimum thread of task, illustrate that then node Ni can't move this task, carry out i=i+1, promptly search next node and whether can satisfy the Capability Requirement of task node.
In step S305, the parallel section of the treatable task of node Ni is dispatched to node Ni.In one embodiment, can not satisfy task to the Capability Requirement of node but greater than the minimum thread of task, then the parallel section with the treatable task of node Ni is dispatched to node Ni when the residue computing power of node Ni.For example, operation task T requires the performed flops of per second to reach 100, its minimum thread is 10 flops, and the residue computing power of node Ni is 20 flops, then the parallel section of 2 threads can be dispatched to node Ni.
In one embodiment, the parallel section of the treatable task of node Ni is dispatched to node Ni after, node Ni can be gone up the task of operation and move.Preferably, be that node Ni is gone up the part task immigration of operation on the nearest node of node Ni, make the residue computing power of node Ni can satisfy the Capability Requirement of task scheduling remaining parallel section after the node i, thereby the remaining parallel section of task is dispatched to node Ni.
In another embodiment, after the parallel section of the treatable task of node Ni is dispatched to node Ni, according to ranking results by near to far searching successively from the nearest node of node Ni (for example node Nj), and task scheduling remaining parallel section behind the node Ni can be dispatched on the node Nj.Should be noted that, when remaining parallel section is dispatched on the node Nj with task, can be with reference to method recited above, further can the residue computing power of decision node Nj satisfy the Capability Requirement of the remaining parallel section of task to node, if can satisfy, then directly the remaining parallel section of task is dispatched to node Nj, otherwise, also further the residue computing power of decision node Nj whether greater than the minimum thread of task, by that analogy, up to the parallel section of task has all been dispatched.Because the node of selected operation task has taken into full account the distance with the pending data of task place node, has effectively reduced the data between nodes transmission, thereby has improved the operational efficiency of supercomputer.
At the remaining parallel section of scheduler task before computing node, can compare to above-mentioned two kinds of volumes of transmitted data that embodiment produced, the volume of transmitted data here is meant the speed of data quantity transmitted and data transmission, therefrom selects the fastest embodiment of data quantity transmitted minimum and data rate that the remaining parallel section of task is dispatched to respective nodes.
The method for scheduling task that should be noted that above-mentioned supercomputer also can be used for the task scheduling in data center, information center, the game center, so the present invention should not be limited to certain particular system.
The above only is preferred embodiment of the present invention, not in order to restriction the present invention, all any modifications of being done within the spirit and principles in the present invention, is equal to and replaces and improvement etc., all should be included within protection scope of the present invention.
Claims (7)
1. the method for scheduling task of a supercomputer is characterized in that, described method comprises:
A. obtain the residue computing power of computing node and the distance between described computing node and the pending data of the task place node;
B. extremely far described computing node is sorted by near by the distance between described computing node and the pending data of the task place node;
C. according among the step B computing node being sorted the ranking results scheduler task that obtains to computing node;
Described step C comprises:
According to described ranking results by near to searching computing node far successively, when the surplus capacity of first computing node does not satisfy task to the Capability Requirement of node, the volume of transmitted data that more following scheduling mode produces: after the parallel section of the treatable task of first computing node is dispatched to first computing node, with the part task immigration that moves on described first computing node on the nearest node of first computing node, satisfy task scheduling remaining parallel section behind first computing node until the residue computing power of described first computing node Capability Requirement of node and is dispatched to the volume of transmitted data that described first computing node is produced with the remaining parallel section of task; The parallel section of the treatable described task of residue computing power of described first computing node is dispatched to first computing node, extremely far find second computing node according to described ranking results by near, and the remaining parallel section of described task is dispatched to the volume of transmitted data that second computing node is produced;
Scheduling mode is made as the less scheduling mode of volume of transmitted data of generation;
Described step C selects data quantity transmitted is little and data rate is the fastest scheduling mode that the remaining parallel section of task that the parallel section of the treatable task of first computing node is dispatched to behind first computing node is dispatched to respective nodes.
2. the method for scheduling task of supercomputer according to claim 1 is characterized in that, described step C comprises:
According to described ranking results by near to searching computing node far successively, when first computing node satisfies task to the Capability Requirement of node, with task scheduling to described computing node.
3. the method for scheduling task of supercomputer according to claim 1 is characterized in that, described step C comprises:
According to described ranking results by near to searching computing node far successively, when the residue computing power of first computing node does not satisfy described task to the Capability Requirement of node, moving on described first computing node of task is moved, residue computing power until described first computing node satisfies the Capability Requirement of task to node, and task scheduling is arrived described first computing node.
4. the method for scheduling task of supercomputer according to claim 1 is characterized in that, described step C comprises:
According to described ranking results by near to searching computing node far successively, when the residue computing power of first computing node does not satisfy described task to the Capability Requirement of node but greater than the minimum thread of described task, the parallel section of the treatable described task of residue computing power of described first computing node is dispatched to first computing node.
5. the method for scheduling task of supercomputer according to claim 4 is characterized in that, described parallel section with task also comprises after being dispatched to first computing node:
Moving on described first computing node of task is moved, can handle the remaining parallel section of described task until the residue computing power of described first computing node, and the remaining parallel section of described task is dispatched to described first computing node.
6. the method for scheduling task of supercomputer according to claim 5 is characterized in that, the described step that moving on first computing node of task is moved is:
With the task immigration that moves on first computing node to the 3rd nearest computing node of described first computing node of distance.
7. the method for scheduling task of supercomputer according to claim 4 is characterized in that, described parallel section with task also comprises after being dispatched to first computing node:
By described ranking results by near to searching second computing node far successively, and described task scheduling remaining parallel section behind first computing node is dispatched to described second computing node.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200910108710 CN101604261B (en) | 2009-07-08 | 2009-07-08 | Task scheduling method for supercomputer |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200910108710 CN101604261B (en) | 2009-07-08 | 2009-07-08 | Task scheduling method for supercomputer |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101604261A CN101604261A (en) | 2009-12-16 |
| CN101604261B true CN101604261B (en) | 2013-07-31 |
Family
ID=41470002
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 200910108710 Active CN101604261B (en) | 2009-07-08 | 2009-07-08 | Task scheduling method for supercomputer |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101604261B (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102289277B (en) * | 2011-07-06 | 2013-12-11 | 中国科学院深圳先进技术研究院 | Dispatching method for data center application services |
| CN104123186B (en) * | 2013-10-15 | 2015-09-16 | 腾讯科技（深圳）有限公司 | Method for distributing business and device |
| CN105721956A (en) * | 2014-12-05 | 2016-06-29 | 中兴通讯股份有限公司 | Frame and communication method |
| CN107025131B (en) * | 2016-02-01 | 2020-12-29 | 腾讯科技（深圳）有限公司 | Task scheduling method and device |
| CN106126323B (en) * | 2016-06-17 | 2019-11-22 | 广州商品清算中心股份有限公司 | Real-time task scheduling method based on cloud platform |
| CN107807796B (en) * | 2017-11-17 | 2021-03-05 | 北京联想超融合科技有限公司 | Data layering method, terminal and system based on super-fusion storage system |
| CN109960573B (en) * | 2018-12-29 | 2021-01-08 | 天津南大通用数据技术股份有限公司 | Cross-domain computing task scheduling method and system based on intelligent perception |
| CN109766189B (en) * | 2019-01-15 | 2022-01-18 | 北京地平线机器人技术研发有限公司 | Cluster scheduling method and device |
| CN110191155B (en) * | 2019-05-07 | 2022-01-18 | 中国人民解放军国防科技大学 | Parallel job scheduling method, system and storage medium for fat tree interconnection network |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101068157A (en) * | 2007-06-07 | 2007-11-07 | 武汉理工大学 | Grid task dispatching method based on time and expense restriction |
| CN101126992A (en) * | 2006-08-15 | 2008-02-20 | 国际商业机器公司 | Method and system for dispensing multiple tasks at multiple node of network |
-
2009
- 2009-07-08 CN CN 200910108710 patent/CN101604261B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101126992A (en) * | 2006-08-15 | 2008-02-20 | 国际商业机器公司 | Method and system for dispensing multiple tasks at multiple node of network |
| CN101068157A (en) * | 2007-06-07 | 2007-11-07 | 武汉理工大学 | Grid task dispatching method based on time and expense restriction |
Non-Patent Citations (1)
| Title |
|---|
| 陈粤等.《基于信号参数集最小距离的并行测试任务调度算法》.《系统仿真学报》.2006,第18卷(第9期),2409-2411. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101604261A (en) | 2009-12-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101604261B (en) | Task scheduling method for supercomputer | |
| US8806498B2 (en) | Method and system for resolving dependency among the enqueued works and/or finished works and scheduling the dependency-resolved works | |
| CN102752198B (en) | Multi-core message forwarding method, multi-core processor and network equipment | |
| CN103309738B (en) | User job dispatching method and device | |
| CN102299843B (en) | Network data processing method based on graphic processing unit (GPU) and buffer area, and system thereof | |
| US9990412B2 (en) | Data driven parallel sorting system and method | |
| CN105512179A (en) | Data sorting device, method and data processing chip achieved by hardware | |
| CN104657214A (en) | Multi-queue multi-priority big data task management system and method for achieving big data task management by utilizing system | |
| CN101162471B (en) | Method and device for insert sorting | |
| CN102882810A (en) | Rapid message transmitting method and device | |
| US7920282B2 (en) | Job preempt set generation for resource management | |
| CN104394096A (en) | Multi-core processor based message processing method and multi-core processor | |
| CN112100233B (en) | Flight time linking method and system based on tabu search algorithm | |
| CN104714838A (en) | Task scheduling method and device | |
| CN104239134A (en) | Method and device for managing tasks of many-core system | |
| Liu et al. | Formal analysis and verification of DDS in ROS2 | |
| CN1829200A (en) | Systems and methods for implementing counters in a network processor | |
| CN101741737B (en) | Maintenance method and device of route table | |
| Liu et al. | On universal scaling of distributed queues under load balancing | |
| CN101196928A (en) | Contents searching method, system and engine distributing unit | |
| Tan et al. | Coupling scheduler for mapreduce/hadoop | |
| KR102032895B1 (en) | Apparatus and method for sharing functional logic between functional units, and reconfigurable processor | |
| US10101965B1 (en) | Method and apparatus for high speed streaming sorter | |
| CN108737455B (en) | Network service identification device and method | |
| CN105471770A (en) | Multi-core-processor-based message processing method and apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20170524 Address after: Room office building No. 1068 Shenzhen Institute of advanced technology A-301 518000 in Guangdong city of Shenzhen province Nanshan District Shenzhen University city academy Avenue Patentee after: Shenzhen shen-tech advanced Cci Capital Ltd Address before: 1068 No. 518055 Guangdong city in Shenzhen Province, Nanshan District City Xili Road School of Shenzhen University Patentee before: Shenzhen Advanced Technology Research Inst. |
|
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20170609 Address after: 4, 8288 building, 66 Shenzhen software Town, Longgang Road, Longgang District, Shenzhen, Guangdong 518000, China Patentee after: Shenzhen Yao Yuan Technology Co., Ltd. Address before: Room office building No. 1068 Shenzhen Institute of advanced technology A-301 518000 in Guangdong city of Shenzhen province Nanshan District Shenzhen University city academy Avenue Patentee before: Shenzhen shen-tech advanced Cci Capital Ltd |
|
| TR01 | Transfer of patent right |