CN106708625A - Minimum-cost maximum-flow based large-scale resource scheduling system and minimum-cost maximum-flow based large-scale resource scheduling method - Google Patents
Minimum-cost maximum-flow based large-scale resource scheduling system and minimum-cost maximum-flow based large-scale resource scheduling method Download PDFInfo
- Publication number
- CN106708625A CN106708625A CN201611121847.0A CN201611121847A CN106708625A CN 106708625 A CN106708625 A CN 106708625A CN 201611121847 A CN201611121847 A CN 201611121847A CN 106708625 A CN106708625 A CN 106708625A
- Authority
- CN
- China
- Prior art keywords
- task
- lowest cost
- maximal flows
- cost
- minimum
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention relates to a minimum-cost maximum-flow based large-scale resource scheduling system and a minimum-cost maximum-flow based large-scale resource scheduling method. The system comprises a task state table, a cluster state table, a scheduling target table, a minimum-cost maximum-flow constructor, a minimum-cost maximum-flow solver and a task executor. The task state table is used for receiving and storing task states submitted by users, and the task states include task CPU (central processing unit) utilization rate, memory utilization rate, network I/O, magnetic disk I/O and priority. The cluster state table is used for storing cluster state information including cluster CPU utilization rate, memory utilization rate and network and magnetic disk I/O and updating cluster states when the cluster state change. The scheduling target table is used for storing scheduling targets configured by the users, and the scheduling targets include priority, placement constraint and fairness currently. The minimum-cost maximum-flow constructor is used for selecting the scheduling targets from the scheduling target table according to information of the task state table and the cluster state table to construct a minimum-cost maximum-flow graph. The minimum-cost maximum-flow solver is used for solving the minimum-cost maximum-flow graph constructed by the minimum-cost maximum-flow constructor according to an incremental algorithm. The task executor is responsible for specific execution of tasks. The minimum-cost maximum-flow based large-scale resource scheduling system and the minimum-cost maximum-flow based large-scale resource scheduling method meet the requirement on flexibility of practical business scenarios.
Description
Technical field
The present invention relates to a kind of extensive resource scheduling system based on maximal flows at lowest cost and method, belong to big data
Resource management field, resource scheduling under particularly extensive environment.
Background technology
With the fast development of the technology such as internet (Internet), Internet of Things (IoT), data (Data) start from simple
Deal with objects to basic Service change, multiple operations (Job) while submit to, executing tasks parallelly (Task) is resolved into, extremely
Operation treatment on few ten thousand grades of physical servers of scale, into the application model of main flow, referred to as " concurrent job
(Concurrentjob) " problem.For example, Github need to process more than 2,000 ten thousand operations every year, Facebook will be responded closely daily
Ten thousand job requests.
Under extensive scheduling of resource refers to multi-objective restriction, task and the optimal mapping decisions process of physical resource, it is solution
The certainly key of concurrent job problem.Existing research work or balance task interference constraints (cross-task interference)
With resource utilization (high resource utilization), such as Whare-Map, Tetrisched；Or balance fairness
(fairness) and data locality (data locality), such as Quincy, YARN；Or balance fairness (fairness) and
Isolation (isolation), such as Mesos, Omega；Or meet the single constraintss such as priority (priority), such as Borg,
Alsched、Quasar.As can be seen here, worked to be typically only capable to be applicable target and immobilized, but correlative study has shown, identical
Operation set be scheduled using different targets, or cause extremely low resource utilization, such as Twitter average resources are utilized
Rate is less than 20%；Or cause transaction capabilities to decline 5 times.Therefore, suitable regulation goal meaning weight is selected according to suitable scene
Greatly.
Fast development and application with big data are goed deep into, and daytime uses fairness dispatch target, are each service class (postal
Part, door etc.) operation offer resource, use priority scheduling target in the evening, preferential to ensure service class operation resource provision, utilization
Slack resources submission big data analysis classes operation (Hadoop, Spark etc.) have turned into a kind of business model of main flow.So produce
Scheduling of resource possesses dynamic object combination, on demand the new demand of configuration take-effective.
The content of the invention
The technology of the present invention solve problem：Overcome the deficiencies in the prior art, it is proposed that a kind of based on maximal flows at lowest cost
Extensive resource scheduling system and method, meet practical business scene flexibility demand, that is, dispatch system and support various scheduling
Target, and different scenes demand can be directed to, optimal scheduling target is flexibly chosen with adaption demand.
The technology of the present invention solution：A kind of extensive resource scheduling system based on maximal flows at lowest cost, including：Appoint
Business state table, cluster state table, regulation goal table, maximal flows at lowest cost constructor, maximal flows at lowest cost solver, task
Actuator, wherein:
Task status table, receives and preserves the task status of user's submission, including the CPU usage of task, internal memory are used
Rate, network I/O, magnetic disc i/o and priority；
Cluster state table, preserves cluster state information, including cluster CPU usage, memory usage, network and disk I/
O, is updated when cluster state changes to cluster state table；
Regulation goal table：The regulation goal of user configuring is stored, at present including priority, placement constraint and fairness；
Maximal flows at lowest cost constructor, according to task status table and cluster state table information, from the selection of regulation goal table
Regulation goal, constructs maximal flows at lowest cost figure；
Maximal flows at lowest cost solver, the most tip constructed to maximal flows at lowest cost constructor using increasable algorithm
Solved with maximum flow graph；
Task performer, is carried out task isolation and is performed, while task status and cluster state are determined using container technique
Phase is synchronized to task status table and cluster state table.
The maximal flows at lowest cost constructor is as follows to the building method of maximal flows at lowest cost figure：
(1) maximal flows at lowest cost model is set up according to maximal flows at lowest cost constructor, there is provided basic operation interface；
(2) task status, cluster state and scheduling mesh are obtained according to task status table, cluster state table, regulation goal table
Mark；
(3) task status, cluster state and scheduling that the basic operation interface and second step for being provided according to the first step are obtained
Target, using different configuration method construct maximal flows at lowest cost figure；
(4) construction result is exported to maximal flows at lowest cost solver.
It is as follows that the use increasable algorithm carries out method for solving to maximal flows at lowest cost figure；
(1) altering event of circulatory monitoring task status table and cluster state table, including addition task, addition machine, will
Change readout maximal flows at lowest cost figure constructor；
(2) maximal flows at lowest cost figure constructor updates maximal flows at lowest cost according to the altering event that the first step is obtained
Figure, the maximal flows at lowest cost figure after renewal is exported to maximal flows at lowest cost solver；
(3) maximal flows at lowest cost solver caching last time solving result, the most tip after the renewal obtained to second step
Local solution is carried out with maximum flow graph, task and resource impact relation is obtained.
A kind of extensive resource regulating method based on maximal flows at lowest cost, comprises the following steps：
Step S01：Task status and cluster state, output to most tip are obtained according to task status table and cluster state table
Use max-flow solver；
Step S02：According to regulation goal table and user configuration information, regulation goal is obtained, output is maximum to least cost
Stream constructor；
Step S03：According to task status, cluster state, regulation goal that step S01 and step S02 is obtained, using minimum
Expense max-flow constructor constructs maximal flows at lowest cost figure, and result is exported to maximal flows at lowest cost solver；
Step S04：According to maximal flows at lowest cost solver, using increasable algorithm, to the minimum constructed in step S03
Expense maximum flow graph is solved, and obtains the mapping relations of task and resource；
Step S05：According to task and resource impact relation that step S04 is obtained, corresponding task is dispatched to particular machine
Task performer on perform.
Present invention advantage compared with prior art is：
(1) present invention proposes the extensive resource regulating method based on maximal flows at lowest cost, supports various regulation goals,
And can on demand switch for different scenes, solve the problems, such as that conventional method is difficult to be applied to various scheduling scenarios；
(2) present invention proposes the incremental SAT algorithm of maximal flows at lowest cost figure, can effectively reduce asking for flow graph
The solution time, make the present invention more with practical value.
Brief description of the drawings
Fig. 1 is the inventive method realization principle figure；
Fig. 2 is maximal flows at lowest cost network model schematic diagram in the inventive method；
Fig. 3 is that maximal flows at lowest cost constructor realizes flow chart in the present invention；
Fig. 4 is that maximal flows at lowest cost solver realizes flow chart in the present invention.
Specific embodiment
To make the present invention easier to understand, with reference to an example, the present invention is further elaborated, but the example not structure
Into any limitation of the invention.
Such as accompanying drawing 1, the technology of the present invention solution：Extensive resource scheduling system based on maximal flows at lowest cost, bag
Include task status table, cluster state table, regulation goal table, maximal flows at lowest cost constructor, maximal flows at lowest cost solver,
Task performer.Wherein:
Task status table is received and preserves the task status of user's submission, including the CPU usage of task, internal memory are used
Rate, network I/O, magnetic disc i/o, priority；
Cluster state table preserves cluster state information, including cluster CPU usage, memory usage, network and disk I/
O, is updated when cluster state changes to cluster state table；
Regulation goal table：The regulation goal of user configuring is stored, at present including priority, placement constraint and fairness；
Maximal flows at lowest cost constructor, believes according in task status table and cluster state table information and regulation goal table
Breath, according to different configuration method, constructs maximal flows at lowest cost figure；
Maximal flows at lowest cost solver is carried out using increasable algorithm to the maximal flows at lowest cost figure that constructor is constructed
Solve；
Task performer, using container technique, carries out tasks carrying and isolates.
As shown in figure 3, the realization of maximal flows at lowest cost constructor is as follows in the present invention：
Maximal flows at lowest cost model is set up using maximal flows at lowest cost constructor first, there is provided basic operation interface；
Then task status, cluster state and regulation goal, output are obtained according to task status table, cluster state table and regulation goal table
To maximal flows at lowest cost constructor；, according to the building method of different regulation goals, construction is most for maximal flows at lowest cost constructor
Small expense max-flow figure.
The maximal flows at lowest cost model and basic operation interface are as follows：
Fig. 2 is maximal flows at lowest cost model, and left side is set of tasks T, and right side is set of physical resources M, side T1,2→M1
And T2,1→U1, represent resource M1And U1It is task T1,2The candidate item of resource provision；The capacity and expense of each edge represent task
Whether schedulable is to (capacity) in respective physical resource, if it can, its supply effect (expense) how.Resource scheduling
After being mapped to minimum cost maximum flow problem, the physical meaning of each element is as shown in table 1 in figure.
Table 1 is based on maximal flows at lowest cost schedule element concrete meaning, and (A → B represents the directed edge that A is constituted to B node, *
Number represent arbitrary node)
The basic operation interface of the figure that the system is provided as shown in table 1, including adds side AddEdge, addition node
AddNode, setting capacity SetCapacity, setup fee SetCost.
The basic operation interface of the figure of table 1
The specific configuration method of the different regulation goals is as follows：
Fairness, constraint, the main regulation goal of three kinds of priority are placed from the capacity and expense that resource visual angle effect is figure
Assignment problem, based on above basic operation interface, introduces fairness, places constraint, the construction algorithm of priority respectively.
1) fairness
Fairness refers to operation set Fairshare physical resource.For example, for Job j, the number of tasks that it is included is Nj, lead to
Cross fair algorithm and calculate its required fair share for Aj.If the resource share that dispatching algorithm distributes to Job j is Aj, then meet public
Levelling.
Fairness is expressed by the capacity construction problem of figure, and algorithm steps are as follows：
I. node U is waited for each operation Job j additionsj
Ii., U is setj→ S capacity bounds are：
[Nj-Aj, Nj-Aj] (1)
Iii. due to UjThe bound of → S is Nj-Aj, then by UjFlow fu=Nj-Aj(maximal flows at lowest cost net
The constraints of network model) it is that waiting for task is needed in Jobj is Nj-AjIt is individual.
The task N of iv.Job jjOr flow to Uj, it is waited for, or flows to resource node being scheduled, therefore Job j
By the flow of resource node
fr=Nj-(Nj-Aj)=Aj (2)
That is Job j are assigned to AjShare resource, meets max-min fairness.
The fairness construction algorithm of the figure of table 2
2) constraint is placed
Place constraint and can be described as the triple as shown in formula (3):
Placement constraint=<Task, resources, utility> (3)
Wherein Task represents task, and resources represents the resource of task restriction, represents that task is placed into resources
On the benefit that is obtained.Constraint is placed using benefit function description, i.e.,
max∑utility (4)
The construction problem that constraint maps to maximal flows at lowest cost network edge and expense is placed, step is as follows：
I. a line task → resources is set up between task and resources in figure
Ii. for newly-established side assigns an expense opposite with benefit value, such as shown in formula (5)
Cost (task → resources)=- utility (5)
Iii. greatest benefit function pair answers least cost
min∑cost (6)
From formula (4), (6), the placement stated by maximal flows at lowest cost network is constrained and is equivalent to place constraint
Definition.
The placement constraint construction algorithm of the figure of table 3
3) priority
In the scheduling for supporting priority, priority priority of task high obtains resource.
Maximal flows at lowest cost network can support strict priority scheduling by expense construction, and step is as follows：
I. Brog is similar to, the expense computing formula with priority scheduling is defined as
Wherein, w1-nRepresent the power per dimension (priority dimension, cpu busy percentage dimension, disk utilization dimension etc.)
Weight, the span specification per dimension is turned to [0, ω], and ω is a constant value.
Ii. it is ∑ priority dimension weight assignment to meet strict priority constraint1≤i≤nwi× ω, then priority compared with
Task expense high is necessarily minimum；
Task correlative charges α is entered as cost (w, v) when iii. being constructed to figure
α=cost (w, v) (8)
The priority construction algorithm of the figure of table 4
As shown in figure 4, maximal flows at lowest cost solver realizes that flow is as follows：
Initialization solution is carried out to figure using existing CostScaling methods first, and preserves solving result；Then, follow
The altering event of ring monitoring task state table and cluster state table, wherein altering event include addition, remove task, addition, shifting
Except resource, cost change and capacity are changed；If monitoring altering event, updated most using maximal flows at lowest cost constructor
Small expense max-flow figure, and maximal flows at lowest cost solver multiplexing last time solving result is triggered, only to the renewal part in figure
Carry out Incremental SAT.
The principle of the Incremental SAT is as follows：
The formalized description of most tip maximum flow networks problem is as follows：
Object function min ∑s(w, v) ∈ EC (w, v) f (w, v) (9)
Constraints
Wherein, w, v represent figure interior joint, and c (w, v) represents the expense of side (w, v), and f (w, v) represents the flow of side (w, v),
U (w, v) represents the capacity of side (w, v), and the summit of b (v) ＞ 0 is source node, and the point of b (v) ＜ 0 is aggregation node.Meet ∑vb
V the stream of ()=0 is referred to as feasible flow.Gesture π (v) of each node v is a real number value, give nodal potential set, side it is relative
Expense (Reduced cost) is defined as cπ(v, w)=c (v, w)-π (v)+π (w).For feasible flow f, G (f) is the remaining of figure G
Network.
When the feasible flow f of minimum cost maximum flow problem meets following condition for the moment, feasible flow f is optimal：
Negative circle optimal condition (Negative cycle optimality)：There is no the increasing that expense is negative in G (f)
Wide circle；
Relative costs optimal condition (Reduced cost optimality)：The relative of each side is taken in G (f)
Use cπ(v, w) is all higher than 0；
Complementary slackness optimal condition (Complementary slackness optimality):The side of G (f), or
Meet cπ(v, w)>0 and f (v, w)=0, or meet 0<F (v, w)<U (v, w) and cπ(v, w)=0, or meet cπ(v, w)<0 and f
(v, w)=u (v, w).
Based on conditions above, have algorithm or safeguard max-flow, iteration reduces expense；Or least cost is safeguarded, iteration increases
Plus stream, i.e., algorithm is that iteration is carried out.According to Google measured results, existing algorithm is adjusted under 100,000 physical resource scales
Degree delay can be more than 90 seconds, without practicality.The present invention proposes a kind of incremental SAT method of figure, by caching and is multiplexed
Last time solving result, it is only necessary to which local operation is carried out to figure, you can the globally optimal solution of figure is obtained, with practicality.
The Incremental SAT specific algorithm is as follows：
For figure G=(V, E, U, C), its minimum cost flow is f, and G network structures locally change, the figure G after change
={ V ', E ', U ', C ' }.Increment type minimum cost flow algorithm, that is, scheme minimum cost flow fs of the G ' based on G and further optimize, and tries to achieve
The minimum cost flow f ' of G '.
First, based on the event-based model circulation global physical resource of inspection state change event Event (task is submitted to,
Machine is delayed machine etc.), when event occurs, the structure of figure is updated, last time solving result is then multiplexed, call Cost scaling to calculate
Method is solved to the figure after renewal.Cost scaling algorithm main thoughts are that continuous iteration finds complementary slackness optimization bar
Part, wherein ∈ represent relaxation condition, and as iterations increase is gradually reduced, f represents initial maximum stream, based on stream f,
CostScaling algorithm iterations find reduce fee condition simultaneously update total cost, until meeting relaxation condition ∈ untill.Increment changes
Generation operation, is updated using push-weight labeling algorithm to relaxation condition.What the time complexity and figure of increasable algorithm changed
Scope is related, is O (VE) under best-case；Need to travel through whole figure under worst case, time complexity is O (V2Elog(VC))。
The increment type min-cost max-flow algorithm of table 4
Extensive resource regulating method of the invention is comprised the following steps：
Step S01：Task status table is received and preserves the task status of user's submission, including task CPU usage, interior
Deposit utilization rate, network I/O, magnetic disc i/o, priority；Cluster state table preservation cluster state information, including cluster CPU usage,
Memory usage, network and magnetic disc i/o, are updated when cluster state changes to cluster state table.Task status and
Cluster state is exported to maximal flows at lowest cost constructor.
Step S02：The regulation goal table of user configuring, one is selected from priority, placement constraint and fairness target
Individual regulation goal.Regulation goal is exported to maximal flows at lowest cost constructor.
Step S03：Maximal flows at lowest cost constructor sets up maximal flows at lowest cost model, there is provided basic operation interface,
And the regulation goal selected according to task status, cluster state and user constructs maximal flows at lowest cost figure.Least cost is maximum
Flow graph is exported to maximal flows at lowest cost solver.
Wherein, to set up process as follows for maximal flows at lowest cost model：
Resource scheduling can be mapped as maximal flows at lowest cost model problem, for model G=(V, E, U, C), resource
Scheduling problem is as follows with its mapping relations：
Set of node V：Resource supply and demand entity is represented, such as resource requirement entity is that task, resource provision entity are physics
Server；
Side collection E：Whether the accessibility between expression task and resource, i.e. task can be mapped to resource.Task and resource
Between exist side be connected, represent it is reachable, otherwise represent unreachable；
Capacity U：The maximum supply of resource is represented, the CPU core number of such as physical server M1 is 12；
Expense C：When task and candidate's provisioning resources are many-one relationships, for assessing every kind of resource supply and demand combination
Effect
Basic operation interface for model G includes：Addition becomes AddEdge, adds node AddNode, sets capacity
SetCapacity, setup fee SetCost.
Different regulation goal building methods are as follows：
Fairness is expressed by the capacity construction problem of figure, and algorithm steps are as follows：
I. node U is waited for each operation Job j additionsj
Ii., U is setj→ S capacity bounds are：
[Nj-Aj, Nj-Aj] (1)
Iii. due to UjThe bound of → S is Nj-Aj, then by UjFlow fu=Nj-Aj(maximal flows at lowest cost net
The constraints of network model) it is that waiting for task is needed in Job j is Nj-AjIt is individual.
The task N of iv.Job jjOr flow to Uj, it is waited for, or flows to resource node being scheduled, therefore Job j
By the flow of resource node
fr=Nj-(Nj-Aj)=Aj (2)
That is Job j are assigned to AjShare resource, meets max-min fairness.
The construction problem that constraint maps to maximal flows at lowest cost network edge and expense is placed, step is as follows：
I. a line task → resources is set up between task and resources in figure
Ii. for newly-established side assigns an expense opposite with benefit value, such as shown in formula (5)
Cost (task → resources)=- utility (5)
Iii. greatest benefit function pair answers least cost
min∑cost (6)
From formula (4), (6), the placement stated by maximal flows at lowest cost network is constrained and is equivalent to place constraint
Definition.
Priority constitution step is as follows：
I. the expense computing formula of priority scheduling is defined as
Wherein, w1-nRepresent the power per dimension (priority dimension, cpu busy percentage dimension, disk utilization dimension etc.)
Weight, the span specification per dimension is turned to [0, ω], and ω is a constant value.
Ii. it is ∑ priority dimension weight assignment1≤i≤nwi× ω, priority task expense higher is necessarily minimum；
Task correlative charges α is entered as cost (w, v) when iii. being constructed to figure
α=cost (w, v) (8)
Step S04：Maximal flows at lowest cost solver uses increment type to the maximal flows at lowest cost figure that step S03 is obtained
Algorithm is solved, and obtains the mapping relations of task and resource.Task is exported to task performer with the mapping relations of resource.
Wherein, increasable algorithm detailed process is as follows,
State change event Event based on the event-based model global physical resource of circulation inspection (submitted to, and machine is delayed by task
Machine etc.), when event occurs, the structure of figure is updated, last time solving result is then multiplexed, Cost scaling algorithms are called to more
Figure after new is solved.Cost scaling algorithm main thoughts are that continuous iteration finds complementary slackness optimal condition, its
Middle ∈ represents relaxation condition, and as iterations increase is gradually reduced, f represents initial maximum stream, based on stream f, CostScaling
Algorithm iteration find reduce fee condition simultaneously update total cost, until meeting relaxation condition ∈ untill.Increment iterative is operated, and is used
Push-weight labeling algorithm is updated to relaxation condition.
Step S05：According to task and resource impact relation that step S04 is obtained, by task scheduling to respective resources, by appointing
Business actuator is responsible for the specific execution of task.Task performer uses container.
Claims (4)
1. a kind of extensive resource scheduling system based on maximal flows at lowest cost, it is characterised in that including：Task status table, collection
Group's state table, regulation goal table, maximal flows at lowest cost constructor, maximal flows at lowest cost solver, task performer, its
In:
Task status table, receive and preserve user submission task status, including task CPU usage, memory usage, net
Network I/O, magnetic disc i/o and priority；
Cluster state table, preserves cluster state information, including cluster CPU usage, memory usage, network and magnetic disc i/o,
Cluster state table is updated when cluster state changes；
Regulation goal table：The regulation goal of user configuring is stored, at present including priority, placement constraint and fairness；
Maximal flows at lowest cost constructor, according to task status table and cluster state table information, from regulation goal table selection scheduling
Target, constructs maximal flows at lowest cost figure；
Maximal flows at lowest cost solver, the least cost constructed to maximal flows at lowest cost constructor using increasable algorithm is most
Big flow graph is solved；
Task performer, is carried out task isolation and is performed, while task status and cluster state is periodically same using container technique
Walk task status table and cluster state table.
2. the extensive resource scheduling system based on maximal flows at lowest cost according to claim 1, it is characterised in that：Institute
The building method that maximal flows at lowest cost constructor is stated to maximal flows at lowest cost figure is as follows：
(1) maximal flows at lowest cost model is set up according to maximal flows at lowest cost constructor, there is provided basic operation interface；
(2) task status, cluster state and regulation goal are obtained according to task status table, cluster state table, regulation goal table；
(3) task status, cluster state and scheduling mesh that the basic operation interface and second step for being provided according to the first step are obtained
Mark, using different configuration method construct maximal flows at lowest cost figure；
(4) construction result is exported to maximal flows at lowest cost solver.
3. the extensive resource scheduling system based on maximal flows at lowest cost according to claim 1, it is characterised in that：Institute
To state using increasable algorithm that carry out method for solving to maximal flows at lowest cost figure as follows；
(1) altering event of circulatory monitoring task status table and cluster state table, including addition task, addition machine, will change
Readout maximal flows at lowest cost figure constructor；
(2) maximal flows at lowest cost figure constructor updates maximal flows at lowest cost figure according to the altering event that the first step is obtained, will
Maximal flows at lowest cost figure after renewal is exported to maximal flows at lowest cost solver；
(3) maximal flows at lowest cost solver caching last time solving result, the least cost after the renewal obtained to second step is most
Big flow graph carries out local solution, obtains task and resource impact relation.
4. a kind of extensive resource regulating method based on maximal flows at lowest cost, its feature is comprised the following steps：
Step S01：Task status and cluster state are obtained according to task status table and cluster state table, is exported to least cost most
Big stream solver；
Step S02：According to regulation goal table and user configuration information, regulation goal, output to maximal flows at lowest cost structure are obtained
Make device；
Step S03：According to task status, cluster state, regulation goal that step S01 and step S02 is obtained, using least cost
Max-flow constructor constructs maximal flows at lowest cost figure, and result is exported to maximal flows at lowest cost solver；
Step S04：According to maximal flows at lowest cost solver, using increasable algorithm, to the least cost constructed in step S03
Maximum flow graph is solved, and obtains the mapping relations of task and resource；
Step S05：According to task and resource impact relation that step S04 is obtained, corresponding task is dispatched to appointing for particular machine
Performed on business actuator.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611121847.0A CN106708625A (en) | 2016-12-08 | 2016-12-08 | Minimum-cost maximum-flow based large-scale resource scheduling system and minimum-cost maximum-flow based large-scale resource scheduling method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611121847.0A CN106708625A (en) | 2016-12-08 | 2016-12-08 | Minimum-cost maximum-flow based large-scale resource scheduling system and minimum-cost maximum-flow based large-scale resource scheduling method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106708625A true CN106708625A (en) | 2017-05-24 |
Family
ID=58936337
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611121847.0A Pending CN106708625A (en) | 2016-12-08 | 2016-12-08 | Minimum-cost maximum-flow based large-scale resource scheduling system and minimum-cost maximum-flow based large-scale resource scheduling method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106708625A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107241752A (en) * | 2017-05-26 | 2017-10-10 | 华中科技大学 | The YARN dispatching methods and system of a kind of sensing network flow |
| CN107423123A (en) * | 2017-07-25 | 2017-12-01 | 广东欧珀移动通信有限公司 | Method for scheduling task, device, storage medium and electronic equipment |
| CN107844999A (en) * | 2017-12-19 | 2018-03-27 | 云南大学 | Network based on auction mechanism hire a car vehicle resources distribution and pricing method |
| CN110515716A (en) * | 2019-08-29 | 2019-11-29 | 中国科学院软件研究所 | It is a kind of to support priority and anti-affine cloud Optimization Scheduling and system |
| CN110532078A (en) * | 2019-08-29 | 2019-12-03 | 中国科学院软件研究所 | A kind of edge calculations method for optimizing scheduling and system |
| CN110995855A (en) * | 2019-12-13 | 2020-04-10 | 深圳先进技术研究院 | Microservice cluster scheduling method, scheduling device and computer readable storage medium |
| CN112601232A (en) * | 2020-12-10 | 2021-04-02 | 中国科学院深圳先进技术研究院 | Load balancing multi-service migration method and system based on minimum cost and maximum flow |
| CN112600827A (en) * | 2020-12-10 | 2021-04-02 | 中国科学院深圳先进技术研究院 | Virtual service migration method and system based on incremental minimum cost maximum flow |
| WO2022062648A1 (en) * | 2020-09-27 | 2022-03-31 | 苏州浪潮智能科技有限公司 | Automatic driving simulation task scheduling method and apparatus, device, and readable medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5524077A (en) * | 1987-07-24 | 1996-06-04 | Faaland; Bruce H. | Scheduling method and system |
| CN102098684A (en) * | 2011-03-22 | 2011-06-15 | 北京邮电大学 | System and method for allocating cross-layer resources in cognitive radio network |
| CN102932279A (en) * | 2012-10-30 | 2013-02-13 | 北京邮电大学 | Multidimensional resource scheduling system and method for cloud environment data center |
-
2016
- 2016-12-08 CN CN201611121847.0A patent/CN106708625A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5524077A (en) * | 1987-07-24 | 1996-06-04 | Faaland; Bruce H. | Scheduling method and system |
| CN102098684A (en) * | 2011-03-22 | 2011-06-15 | 北京邮电大学 | System and method for allocating cross-layer resources in cognitive radio network |
| CN102932279A (en) * | 2012-10-30 | 2013-02-13 | 北京邮电大学 | Multidimensional resource scheduling system and method for cloud environment data center |
Non-Patent Citations (2)
| Title |
|---|
| 梁大桥: "MapReduce作业调度优化技术研究", 《万方数据知识服务平台》 * |
| 陈晓旭等: "基于最小费用最大流的大规模资源调度方法", 《软件学报》 * |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107241752B (en) * | 2017-05-26 | 2019-10-25 | 华中科技大学 | A kind of the YARN dispatching method and system of sensing network flow |
| CN107241752A (en) * | 2017-05-26 | 2017-10-10 | 华中科技大学 | The YARN dispatching methods and system of a kind of sensing network flow |
| CN107423123A (en) * | 2017-07-25 | 2017-12-01 | 广东欧珀移动通信有限公司 | Method for scheduling task, device, storage medium and electronic equipment |
| CN107844999B (en) * | 2017-12-19 | 2021-06-04 | 云南大学 | Auction mechanism-based network vehicle renting resource allocation and pricing method |
| CN107844999A (en) * | 2017-12-19 | 2018-03-27 | 云南大学 | Network based on auction mechanism hire a car vehicle resources distribution and pricing method |
| CN110515716A (en) * | 2019-08-29 | 2019-11-29 | 中国科学院软件研究所 | It is a kind of to support priority and anti-affine cloud Optimization Scheduling and system |
| CN110532078A (en) * | 2019-08-29 | 2019-12-03 | 中国科学院软件研究所 | A kind of edge calculations method for optimizing scheduling and system |
| CN110515716B (en) * | 2019-08-29 | 2021-11-30 | 中国科学院软件研究所 | Cloud optimization scheduling method and system supporting priority and inverse affinity |
| CN110995855A (en) * | 2019-12-13 | 2020-04-10 | 深圳先进技术研究院 | Microservice cluster scheduling method, scheduling device and computer readable storage medium |
| CN110995855B (en) * | 2019-12-13 | 2022-06-21 | 深圳先进技术研究院 | Microservice cluster scheduling method, scheduling device and computer readable storage medium |
| WO2022062648A1 (en) * | 2020-09-27 | 2022-03-31 | 苏州浪潮智能科技有限公司 | Automatic driving simulation task scheduling method and apparatus, device, and readable medium |
| US11868808B2 (en) | 2020-09-27 | 2024-01-09 | Inspur Suzhou Intelligent Technology Co., Ltd. | Automatic driving simulation task scheduling method and apparatus, device, and readable medium |
| CN112601232A (en) * | 2020-12-10 | 2021-04-02 | 中国科学院深圳先进技术研究院 | Load balancing multi-service migration method and system based on minimum cost and maximum flow |
| CN112600827A (en) * | 2020-12-10 | 2021-04-02 | 中国科学院深圳先进技术研究院 | Virtual service migration method and system based on incremental minimum cost maximum flow |
| CN112600827B (en) * | 2020-12-10 | 2021-10-29 | 中国科学院深圳先进技术研究院 | Virtual service migration method and system based on incremental minimum cost maximum flow |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106708625A (en) | Minimum-cost maximum-flow based large-scale resource scheduling system and minimum-cost maximum-flow based large-scale resource scheduling method | |
| Yang et al. | Delay-aware virtual network function placement and routing in edge clouds | |
| Sun et al. | Re-Stream: Real-time and energy-efficient resource scheduling in big data stream computing environments | |
| Pacini et al. | Distributed job scheduling based on Swarm Intelligence: A survey | |
| Zhang et al. | Network-aware virtual machine migration in an overcommitted cloud | |
| Addya et al. | Simulated annealing based VM placement strategy to maximize the profit for Cloud Service Providers | |
| Hao et al. | An enhanced load balancing mechanism based on deadline control on GridSim | |
| Gao et al. | An energy-aware ant colony algorithm for network-aware virtual machine placement in cloud computing | |
| Tziritas et al. | On minimizing the resource consumption of cloud applications using process migrations | |
| De Coninck et al. | Dynamic auto-scaling and scheduling of deadline constrained service workloads on IaaS clouds | |
| Wang et al. | Joint server assignment and resource management for edge-based MAR system | |
| Zhang et al. | Adaptive multi-objective artificial immune system based virtual network embedding | |
| Chen et al. | Scheduling independent tasks in cloud environment based on modified differential evolution | |
| Li et al. | An effective scheduling strategy based on hypergraph partition in geographically distributed datacenters | |
| Mirzayi et al. | A hybrid heuristic workflow scheduling algorithm for cloud computing environments | |
| Ding et al. | Kubernetes-oriented microservice placement with dynamic resource allocation | |
| Gu et al. | Maximizing workflow throughput for streaming applications in distributed environments | |
| Xia et al. | Efficient data placement and replication for QoS-aware approximate query evaluation of big data analytics | |
| Zhang et al. | Design and implementation of task scheduling strategies for massive remote sensing data processing across multiple data centers | |
| Chen et al. | Heterogeneous semi-asynchronous federated learning in Internet of Things: A multi-armed bandit approach | |
| Keerthika et al. | A multiconstrained grid scheduling algorithm with load balancing and fault tolerance | |
| CN108304253A (en) | Map method for scheduling task based on cache perception and data locality | |
| Priya et al. | To optimize load of hybrid P2P cloud data-center using efficient load optimization and resource minimization algorithm | |
| Xia et al. | Proactive and intelligent evaluation of big data queries in edge clouds with materialized views | |
| Kumar et al. | QoS‐aware resource scheduling using whale optimization algorithm for microservice applications |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20170524 |