CN106228314A - The workflow schedule method of study is strengthened based on the degree of depth - Google Patents

The workflow schedule method of study is strengthened based on the degree of depth Download PDF

Info

Publication number
CN106228314A
CN106228314A CN201610656579.6A CN201610656579A CN106228314A CN 106228314 A CN106228314 A CN 106228314A CN 201610656579 A CN201610656579 A CN 201610656579A CN 106228314 A CN106228314 A CN 106228314A
Authority
CN
China
Prior art keywords
task
dag
directed acyclic
workflow
acyclic graph
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201610656579.6A
Other languages
Chinese (zh)
Inventor
段翰聪
闵革勇
张建
王瑾
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Electronic Science and Technology of China
Original Assignee
University of Electronic Science and Technology of China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Electronic Science and Technology of China filed Critical University of Electronic Science and Technology of China
Priority to CN201610656579.6A priority Critical patent/CN106228314A/en
Publication of CN106228314A publication Critical patent/CN106228314A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G06Q10/063—Operations research, analysis or management
    • G06Q10/0631—Resource planning, allocation, distributing or scheduling for enterprises or organisations
    • G06Q10/06313—Resource planning in a project environment
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Human Resources & Organizations (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Strategic Management (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Economics (AREA)
  • Entrepreneurship & Innovation (AREA)
  • General Business, Economics & Management (AREA)
  • Biomedical Technology (AREA)
  • Operations Research (AREA)
  • Quality & Reliability (AREA)
  • Tourism & Hospitality (AREA)
  • Game Theory and Decision Science (AREA)
  • Educational Administration (AREA)
  • Development Economics (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Marketing (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The invention discloses the workflow schedule method strengthening study based on the degree of depth, comprise the steps: step A) the tasks carrying DAG workflow directed acyclic graph M that collects in actual execution environment opens, as sample pool；Step B) every DAG workflow directed acyclic graph is carried out MDP markov decision process modeling, generate task status set S；Step C) according to the training method DQN of neutral net, M is opened task status set S and corresponding known action set A that DAG workflow directed acyclic graph generates as input, substitute into deep neural network formula, try to achieve neural network parameter matrixValue.The present invention passes through said method, it is long that the workflow schedule method under current distributed environment that solves performs the time, and the defect of generalization difference accelerates to ensure the time efficiency of algorithm, simultaneously increase algorithm itself Generalization Capability, allow dispatch machine can be according to actual scene feature autonomic learning scheduling strategy.

Description

The workflow schedule method of study is strengthened based on the degree of depth
Technical field
The present invention relates to computer software fields, in particular it relates to strengthen the workflow schedule method of study based on the degree of depth.
Background technology
In a distributed computing environment, workflow schedule problem (workflow scheduling) is the most all meter One of optimization problem in calculation machine field.Workflow schedule problem actually provides a scheduling scheme, by workflow Task be dispatched in a certain order properly to perform on node perform, to minimize Executing Cost.Its mathematical model is such as Under:
One concrete calculating application can be represented by a directed acyclic graph (DAG) G (T, E), and wherein T is n Business set { t1,t2,...,tn, E is the dependence set between task.(i, j) ∈ E represents task to each dependence e tjNeed in task tiExecution could start after completing to perform.Providing limited collection of machines M, M comprises n platform node {m1,m2,...,mn}.Make χ represent all of distribution condition set, typically can be expressed as element x ∈ χ a | T | × | M | matrix, represent a kind of plan of distribution, meanwhile, it is assumed that there are cost function C: χ → [0 ,+a ∞], cost function can To be represented as always performing time (makespan) length, then be furnished with for each component: Cij: M × J → [0 ,+∞] is wherein CijExpression task JiIt is assigned to machine MjThe cost of rear execution.To sum up, job shop scheduling problem can abstract be, obtain a component and join Solving x ∈ χ makes C (x) minimum, say, that there is not a component and joins solution y and make C (y) < C (x).So, for job scheduling Optimization problem, has two main aspects to need to consider, one is the occupancy of system resource, and one is that whole Job execution completes Total time.
Workflow schedule problem has been proved to belong to np complete problem, it is impossible to find the solution of polynomial time complexity. Existing solution np complete problem generally uses heuritic approach, genetic algorithm or Q learning algorithm, and heuritic approach can only calculate secondary Excellent solution；Genetic algorithm needs to carry out iterative process for several times, and just can finding compares preferably solves, when whole iterative process performs Between longer, calculate in real time on platform in big data, the time loss of optimization even cannot accept；Q learning algorithm is general In the property changed and performance all well, but once number of tasks magnitude is excessive, and whole state number is too much, Q-value matrix dimension mistake Height causes computer cannot store whole Q-value matrix (Q (s, a)).
Summary of the invention
Instant invention overcomes the deficiencies in the prior art, it is provided that strengthen the workflow schedule method of study based on the degree of depth, solve Under distributed environment, the workflow schedule method execution time is long at present, the defect of generalization difference, accelerates to ensure the time effect of algorithm Rate, simultaneously increase algorithm itself Generalization Capability, allow dispatch machine can be according to actual scene feature autonomic learning scheduling strategy.
The present invention solves the problems referred to above and be the technical scheme is that the workflow schedule side strengthening study based on the degree of depth Method, comprises the steps:
Step A) the tasks carrying DAG workflow directed acyclic graph M that collects in actual execution environment opens, as sample pool；
Step B) every DAG workflow directed acyclic graph is carried out MDP markov decision process modeling, generate task shape State set S；
Step C) according to the training method DQN of neutral net, M is opened the task shape that DAG workflow directed acyclic graph generates State set S and corresponding known action set A, as input, substitutes into deep neural network formula Q (s, a；θi), try to achieve execution and appoint Neural network parameter matrix θ during business iiValue, Q is value of the movements function, and s is in task status set S, and a is Make a scheduling scheme in set A；
Step D) judge that the task status set S that DAG workflow directed acyclic graph generates substitutes into nerve the most successively The training method DQN of network, if all substituted into, then exports final neural network parameter matrix θiValue, without then Continue executing with the training method DQN of neutral net.
Step E) for newly inputted DAG workflow directed acyclic task, carry out MDP markov decision process equally and build Mould, generate initiating task state S0, substitute into step D) in deep neural network formula Q (s, a；θi), wherein θiValue be step D) calculated value in, can obtain a scheduling scheme a in final set of actions A, and will input DAG work specifically Task status and the scheduling result of making stream directed acyclic graph are input to sample pool.
This programme the i.e. degree of depth of Q learning algorithm strengthen study on the basis of, introduce deep neural network substitute Q (s, a) (s, a) wherein θ is the parameter matrix of neutral net to i.e. Q (s, a, θ) ≈ Q, and θ is that the parameter matrix of neutral net uses neutral net Training method DQN be calculated minima, and s is obtained by mathematical modeling, calculates according to the training method DQN of neutral net Obtain minima θ and s substitutes into Q (s, a, θ) formula, i.e. can get optimal scheduling scheme a.A described tool in this programme The deep neural network of body is illustrated in fig. 2 shown below, and neutral net comprises a lot of level, and each level comprises some neurons, one Neuron substantially can be described as function, receives the output of upper strata neuron, calculates through function, exports to lower floor neural Unit.The function that inside neurons uses, we term it activation primitive, typically we use ReLu (Rectified Linear Units) activation primitive, specific as follows shown in, that input vector s, s are in MDP modeling in task status set, We have
F (θ s)=max (0, θ s)
For a neuron in a layer, θ is substantially a 1*n vector.According to neutral net this one One mapping relations, after we obtain θ value, can find the value of the Q (s, a, θ) of correspondence, the s generation then obtained according to mathematical modeling Enter and obtain optimal scheduling scheme a.This scheduling scheme a makes the task in workflow be dispatched in a certain order properly perform Perform on node, make Executing Cost minimum.The method avoids the occurrence of in Q learning algorithm because status number S and set of actions A may Very big, the defect that often computer can not be left.
Technical solution of the present invention i.e. uses same model to different DAG in scheduling unlike common heuritic approach Input is scheduling, and also unlike genetic algorithm, carries out successive ignition execution during calculating scheduling result, consumes a large amount of Time.Meanwhile, use the most merely Q learning algorithm, but use DQN (Q learning algorithm combines deep neural network) to carry out Workflow schedule.Owing to calculating based on deep neural network can accelerate to ensure the time efficiency of algorithm by GPU, increase simultaneously The Generalization Capability of computation system itself, allow scheduling machine can be according to actual scene feature autonomic learning scheduling strategy.
It is further preferred that also include step F) after sample pool accumulation to a certain extent, repeat step C) to the degree of depth Neutral net formula Q (s, a；θi) calculate again, obtain new θiValue and new deep neural network Q ' (s, a；θi) be used for after The DAG workflow directed acyclic graph scheduling of continuous input calculates.
Sample pool is accumulated to a certain degree to refer to the sample number of the DAG workflow directed acyclic graph accumulated in sample pool Amount, more than 100, just starts step C) calculate, training sample random can sample 100 DAG workflows from sample pool Directed acyclic graph calculates.Along with being continuously increased of DAG workflow directed acyclic graph in sample pool, utilize the instruction of neutral net Practice method DQN, constantly update θiValue after, then carry out being calculated optimal scheduling scheme, allow the scheduling machine can basis Actual scene feature autonomic learning scheduling strategy.
Preferably, step C) according to markov decision process release neutral net training method DQN step such as Under:
Step C1) for each time point t, there is markoff process et=(st,at,rt,st+1), define storage pool D =e1,e2…eN, initializing storage pool D size is N, initializes and is worth function of movement Q, and gradient descent method iterations is M time, M For DAG workflow directed acyclic graph number, initialization task state set stInThereinIt is s1Corresponding mapping function, thenIt is stCorresponding mapping function；
Step C2) after performing a period of time, obtain the scheduling scheme a of corresponding time point ttIf scheduling scheme is only One is then atItself, otherwiseObtain the task status of task status set (t+1) time point SetBy nowIt is stored in storage pool D；
Step C3) to make i be DAG workflow directed acyclic graph number in storage pool, if last DAG that i is circulation Workflow directed acyclic graph, then make yi=ri, riFor feedback function R (St,at) value in corresponding moment；If i is not to circulate A rear DAG workflow directed acyclic graph, then make riFor feedback function R (St, at) value in corresponding moment,It is stCorresponding mapping function, a ' is the scheduling scheme of corresponding time point t, and Υ is decay system Number, the span of attenuation quotient is 0-1.
Step C4) according to gradient descent algorithm, the loss function L to deep neural networki(θi)=ES, a～p ()[(yi-Q(s,a；θi))2], Iteration M time, i.e. seeks local derviation to this loss function Try to achieve the minima of this loss function, and the minima of this loss function is substituted into formulaTo θiEnter Row updates, and wherein α is called learning rate is constant, and here learning rate value is 0.001, until θiConvergence or iteration complete into Only, it is possible to obtain the parameter matrix θ of neutral netiValue.
Step B) in that every DAG workflow directed acyclic graph is carried out MDP markov decision process modeling procedure is as follows:
Step B1) store DAG workflow directed acyclic graph with two-dimensional array G (T, E), wherein T represents that DAG workflow has Task node set in acyclic figure, E represents the limit collection in DAG workflow directed acyclic graph；
Step B2) definition (Ti,Tj) ∈ E represents that task j must perform to perform afterwards in task i, defines V (Ti,Ti) ＞ 0 represents task TiThe execution time of estimating of itself, define V (Ti,Tj) represent that if task i and task j are assigned to different The call duration time that node performs, as V (Ti, TjDependence is not had between)=∞ explanation task i and task j；
Step B3) by the two-dimensional matrix V of N × (N+1)N×(N+1)Represent distribution state S of whole workflow, The sum of task node during wherein N represents DAG, the last string i.e. N+1 row of matrix, represent the distribution condition of each task；
Step B4) definition feedback functionWherein t (St) represent task status StUnder the time that always performs. C represents original state S0Under the time that always performs,
To sum up, the invention has the beneficial effects as follows:
This programme i.e. uses same model to carry out different DAG inputs in scheduling unlike common heuritic approach Scheduling, also unlike genetic algorithm, carries out successive ignition execution during calculating scheduling result, consumes the plenty of time, adopt Combine deep neural network with Q learning algorithm and be operated stream scheduling, accelerate to ensure the time efficiency of algorithm, increase calculation simultaneously The Generalization Capability of method itself, allow scheduling machine can be according to actual scene feature autonomic learning scheduling strategy.
Accompanying drawing explanation
Fig. 1 is the flow chart that the present invention runs based on deep neural network；
Fig. 2 be the present invention Q (s, a,；θ) deep neural network；
Fig. 3 is the DAG task scheduling figure of the present invention.
Detailed description of the invention
Below in conjunction with embodiment and accompanying drawing, to the detailed description further of present invention work, but embodiments of the present invention It is not limited to this.
The DAG mentioned in this programme: i.e. workflow directed acyclic graph；MDP: markov decision process；
DQN: degree of depth Q-value network, is that the degree of depth strengthens the core learnt, is substantially a kind of deep neural network, be used for replacing For Q-value matrix noted earlier.
Embodiment 1:
As Figure 1-3, the present invention includes the workflow schedule method strengthening study based on the degree of depth, comprises the steps:
Step A) the tasks carrying DAG workflow directed acyclic graph M that collects in actual execution environment opens, as sample pool；
Step B) every DAG workflow directed acyclic graph is carried out MDP markov decision process modeling, generate task shape State set S；
Step C) according to the training method DQN of neutral net, M is opened the task shape that DAG workflow directed acyclic graph generates State set S and corresponding known action set A, as input, substitutes into deep neural network formula Q (s, a；θi), try to achieve execution and appoint Neural network parameter matrix θ during business iiValue, Q is value of the movements function, and s is in task status set S, and a is Make a scheduling scheme in set A；
Step D) judge that the task status set S that DAG workflow directed acyclic graph generates substitutes into nerve the most successively The training method DQN of network, if all substituted into, then exports final neural network parameter matrix θiValue, without then Continue executing with the training method DQN of neutral net.
Step E) for newly inputted DAG workflow directed acyclic task, carry out MDP markov decision process equally and build Mould, generate initiating task state S0, substitute into step D) in deep neural network formula Q (s, a；θi), wherein θiValue be step D) calculated value in, can obtain a scheduling scheme a in final set of actions A, and will input DAG work specifically Task status and the scheduling result of making stream directed acyclic graph are input to sample pool.
Step C) in the training method DQN step of neutral net released according to markov decision process as follows:
Step C1) for each time point t, there is markoff process et=(st,at,rt,st+1), define storage pool D =e1,e2…eN, initializing storage pool D size is N, initializes and is worth function of movement Q, and gradient descent method iterations is (1-M) Secondary, M is DAG workflow directed acyclic graph number, initialization task state set stInWherein 'sIt is s1Corresponding mapping function, thenIt is stCorresponding mapping function；
Step C2) after performing a period of time, obtain the scheduling scheme a of corresponding time point ttIf scheduling scheme is only One is then atItself, otherwiseObtain the task status of task status set (t+1) time point SetBy nowIt is stored in storage pool D；
Step C3) to make i be DAG workflow directed acyclic graph number in storage pool, if last DAG that i is circulation Workflow directed acyclic graph, then make yi=ri, riFor feedback function R (St,at) value in corresponding moment；If i is not to circulate A rear DAG workflow directed acyclic graph, then make riFor feedback function R (St, at) value in corresponding moment,It is stCorresponding mapping function, a ' is the scheduling scheme of corresponding time point t, and Υ is decay system Number, the span of this attenuation quotient is between 0 to 1.
Step C4) according to gradient descent algorithm, the loss function L to deep neural networki(θi)=ES, a～p ()[(yi-Q(s,a；θi))2], Iteration M time, i.e. seeks local derviation to this loss function Try to achieve the minima of this loss function, and the minima of this loss function is substituted into formulaTo θiEnter Row updates, and wherein α is called learning rate is constant, and here learning rate value is 0.001, until θiConvergence or iteration complete into Only, it is possible to obtain the parameter matrix θ of neutral netiValue.
Step B) in that every DAG workflow directed acyclic graph is carried out MDP markov decision process modeling procedure is as follows:
Step B1) store DAG workflow directed acyclic graph with two-dimensional array G (T, E), wherein T represents that DAG workflow has Task node set in acyclic figure, E represents the limit collection in DAG workflow directed acyclic graph；
Step B2) definition (Ti,Tj) ∈ E represents that task j must perform to perform afterwards in task i, defines V (Ti,Ti) ＞ 0 represents task TiThe execution time of estimating of itself, define V (Ti,Tj) represent that if task i and task j are assigned to different The call duration time that node performs, as V (Ti,TjDependence is not had between)=∞ explanation task i and task j；
Step B3) by the two-dimensional matrix V of N × (N+1)N×(N+1)Represent distribution state S of whole workflow, its Middle N represents the sum of task node in DAG, the last string i.e. N+1 row of matrix, represents the distribution condition of each task；
Step B4) definition feedback functionWherein t (St) represent task status StUnder the time that always performs. C represents original state S0Under the time that always performs,
The scheduling of work on hand stream generally uses following algorithm to solve: is respectively heuritic approach, genetic algorithm and Q study and calculates Method.
The first uses heuritic approach to solve workflow schedule problem is the classical dispatching algorithm of comparison.List First scheduling algorithm can calculate the priority of each task according to certain algorithm, and then determining according to priority should Which task this dispatches after first dispatching which task.Substantially, this class algorithm is undertaken in two steps, and the first step obtains all tasks Priority, second step is task Resources allocation to be scheduled.The priority meter that different List scheduling algorithms uses Calculation method and resource selection method are all not quite similar.
Use List scheduling Algorithm for Solving time complexity typically at O (n2), speed is relatively fast.But calculate Method itself belongs to heuritic approach, can only obtain suboptimal solution.And algorithm Generalization Capability is poor, and scheduling scenario once becomes Changing, the result that its algorithm is obtained is not necessarily well solution.
The second uses genetic algorithm to contrast in generalization and to get well with common heuritic approach, has the strongest general Property, but its shortcoming is it is also obvious that genetic algorithm needs to carry out iterative process for several times, and just can finding compares preferably solves, whole It is longer that individual iterative process performs the time, calculates in real time on platform in big data, and the time loss of optimization even cannot accept 's.
The third uses Q learning algorithm (Q-Learning), for the Mission Scheduling in collaborative work, establishes phase The Markovian decision process model answered, proposes the Q learning algorithm based on simulated annealing of a kind of improvement on this basis.Should Algorithm is by introducing simulated annealing, and combines greedy strategy, and the screening on state space judges, considerably improves receipts Hold back speed, shorten the execution time.Utilize Q learning algorithm to be trained obtaining detailed protocol and see " task scheduling based on Q study The linguistic term of problem " (graphics journal in March, 2012).
The program make use of the technology strengthening study, carries out abstract to DAG scheduling problem.Algorithm is at generalization and performance table Now going up all good, but once number of tasks magnitude is excessive, whole state number is too much, and Q-value matrix dimension is too high causes computer Whole Q-value matrix (Q (s, a)) cannot be stored.
Degree of depth enhancing study is degree of depth study to be combined with enhancing study thus realizes perceiving from Perception A kind of brand-new algorithm of the end-to-end study of Action action.It is the best that degree of depth enhancing study has in terms of solving decision problem Effect, it possesses makes robot realize the most entirely autonomous potentiality learning the most multiple a kind of technical ability.
The degree of depth strengthens learning theory basis and is markov decision process.One typical markov decision process By four-tuple<S, A, P, R, a γ>composition, wherein S represents that state set, A represent that set of actions, P represent State Transferring Probability matrix, R represents backoff values, and γ is attenuation quotient, and span is 0 to 1.
Define a decision-making π (a | s)=P [At=a | St=s] represent state StSelection action AtProbability, definition action Cost function (action-value) Qπ(s, a)=Eπ[Gt|St=s, At=a] wherein GtRepresent accumulated value,Wherein subscript t express time, k is constant value.Eπ[Gt|St=s, At=a] represent shape under strategy π State s makes the expectation of the accumulated value that action a is obtained.The problem so strengthening Learning demands solution is exactly, and determines all of Plan is found the value of the movements function of maximum value i.e.:
Q (s, a)=maxπQπ(s,a)；
Above formula is carried out expansion can obtain:
Q (s, a)=maxπEπ[Rt|St=s, At=a]；
This problem is solved itself to solve with dynamic programming, above-mentioned formula is converted into following recurrence Equation represents:
Q (s, a)=maxπES '～ε[R+γmaxa, Q (s ', a ') | s, a]；
For solving this recursion equation, can to use in the way of dynamic programming, i.e. store each Q (s, value a), but Actually, this realization is unpractical, because status number S and set of actions A may be very big, often computer can not Leave.
So this programme is on the basis of the i.e. degree of depth of Q learning algorithm strengthens study, introduces deep neural network and substitute Q (s is a) that (s, a) wherein θ is the parameter matrix of neutral net to Q (s, a, θ) ≈ Q, and θ is that the parameter matrix of neutral net uses god It is calculated minima through the training method DQN of network, and s is obtained by mathematical modeling, according to the training method of neutral net DQN is calculated minima θ and s substitutes into Q (s, a, θ) formula, i.e. can get optimal scheduling scheme a.Described in this programme A concrete deep neural network be illustrated in fig. 2 shown below, neutral net comprises a lot of level, and each level comprises some nerves Unit, a neuron substantially can be described as function, receives the output of upper strata neuron, calculates through function, and output is given down Layer neuron.The function that inside neurons uses, we term it activation primitive, typically we use ReLu (Rectified Linear Units) activation primitive, specific as follows shown in, input vector s, s are in MDP modeling in task status set One, Wo Menyou
F (θ s)=max (0, θ s)
For a neuron in a layer, θ is substantially a 1*n vector.According to neutral net this one One mapping relations, after we obtain θ value, can find the value of the Q (s, a, θ) of correspondence, the s generation then obtained according to mathematical modeling Enter and obtain optimal scheduling scheme a.This scheduling scheme a makes the task in workflow be dispatched in a certain order properly perform Perform on node, make Executing Cost minimum.The method avoids the occurrence of in Q learning algorithm because status number S and set of actions A may Very big, the defect that often computer can not be left.
Technical solution of the present invention i.e. uses same model to different DAG in scheduling unlike common heuritic approach Input is scheduling, and also unlike genetic algorithm, carries out successive ignition execution during calculating scheduling result, consumes a large amount of Time.Meanwhile, use the most merely Q learning algorithm, but use DQN (Q learning algorithm combines deep neural network) to carry out Workflow schedule.Owing to calculating based on deep neural network can accelerate to ensure the time efficiency of algorithm by GPU, simultaneously Increase algorithm itself Generalization Capability, allow dispatch machine can be according to actual scene feature autonomic learning scheduling strategy.
When being operated stream scheduling, initialized state S0 of DAG to be scheduled by MDP modeling process；By state S0 As the input of deep neural network, (s a), chooses the scheduling scheme that Q-value is maximum, updates DAG shape to obtain scheduling strategy Q-value Q State；Whether disconnected DAG state is to have dispatched all tasks, if having dispatched all tasks, output scheduling result, without so Continue iterative process.
Such as Fig. 3, table 1, table 2, table 3, it is simply that use the method described in this programme, by DAG workflow directed acyclic task image Corresponding task T1～T9 are assigned in 4 node M 1～M4 perform:
As Figure of description 3 show DAG task scheduling figure, inside circle, represent execution task and execution task needs Time, arrow represents next step direction performed, the passing time between digitized representation two task on arrow.Such as: task T1 oneself the execution time requires time for 2 seconds, and task T2 that is delivered to after being finished requires time for 4 seconds, is delivered to task T3, T4 Being required to 1 second with T5, being delivered to T7 needs 10 seconds, and T2 oneself is finished and needs 3 seconds, is delivered to task after being finished again T6, by that analogy, performs with this, until having performed task T9.
State S0 before unallocated task, matrix VN×NIt is as shown in table 1 below, under wherein C=66 is original state S0 Always perform the time, between two tasks of digitized representation corresponding between abscissa and vertical coordinate, perform the required time, as sat Mark T1T1=2 represents T1 oneself and performs to require time for 2 seconds, and by that analogy, M row represent whether node performs, and 1 represents also do not have Performing, 1 representative has performed；
After scheduler task T1 to node M 1, task status S1 attempt to change into, its backoff values is 66/66=1, such as table 2 below Shown in；
Then scheduling T2 performs to node M 1, and task status S2 figure is shown in table 3 below, calculates backoff values and is: 66/62= 1.06；
Owing to T1 and T2 performs on same node, so V (T1, T2) value becomes 0 that is task 1 and task 2 The call duration time expense performed becomes 0.
After model has been set up, just had input Q as DQN algorithm (s, a), then determine for such a Model, learns us by successive ignition and can train our degree of depth Q network, accelerates to ensure the time efficiency of algorithm, simultaneously Increase algorithm itself Generalization Capability, allow dispatch machine can be according to actual scene feature autonomic learning scheduling strategy.
Embodiment 2:
The present embodiment is preferably as follows on the basis of embodiment 1: also include step F) when sample pool accumulate to a certain extent Afterwards, step C is repeated) to deep neural network formula Q (s, a；θi) calculate again, obtain new θiValue and new degree of depth god Through network Q (s, a；θi) for follow-up input DAG workflow directed acyclic graph scheduling calculate.
Sample pool is accumulated to a certain degree to refer to the sample number of the DAG workflow directed acyclic graph accumulated in sample pool Amount, more than 100, just starts step C) calculate, training sample random can sample 100 DAG workflows from sample pool Directed acyclic graph calculates.
Along with being continuously increased of DAG workflow directed acyclic graph in sample pool, utilize the training method DQN of neutral net, Constantly update θiValue after, then carry out being calculated optimal scheduling scheme, allow the scheduling machine can be special according to actual scene Levy autonomic learning scheduling strategy.
The above, be only presently preferred embodiments of the present invention, and the present invention not does any pro forma restriction, every depends on Any simple modification of being made above example according to the technical spirit of the present invention, equivalent variations, each fall within the protection of the present invention Within the scope of.

Claims (5)

1. strengthen the workflow schedule method of study based on the degree of depth, it is characterised in that comprise the steps:
Step A) the tasks carrying DAG workflow directed acyclic graph M that collects in actual execution environment opens, as sample pool；
Step B) every DAG workflow directed acyclic graph is carried out MDP markov decision process modeling, generate task status collection Close S；
Step C) according to the training method DQN of neutral net, M is opened the task status collection that DAG workflow directed acyclic graph generates Close S and corresponding known action set A as input, substitution deep neural network formula Q (s, a；θi), try to achieve execution task i Time neural network parameter matrix θiValue, Q is value of the movements function, and s is in task status set S, and a is behavior aggregate Close a scheduling scheme in A；
Step D) judge that the task status set S that DAG workflow directed acyclic graph generates substitutes into neutral net the most successively Training method DQN, if all substituted into, then export final neural network parameter matrix θiValue, without then continuing Perform the training method DQN of neutral net；
Step E) for newly inputted DAG workflow directed acyclic task, carry out MDP markov decision process modeling equally, Generate initiating task state S0, substitute into step D) in deep neural network formula Q (s, a；θi), wherein θiValue be step D) In calculated value, can obtain a scheduling scheme a in final set of actions A, and DAG work will be inputted specifically Task status and the scheduling result of stream directed acyclic graph are input to sample pool.
The workflow schedule method strengthening study based on the degree of depth the most according to claim 1, it is characterised in that also include step Rapid F) after sample pool accumulation to a certain extent, repeat step C) to deep neural network formula Q (s, a；θi) count again Calculate, obtain new θiValue and new deep neural network Q ' (s, a；θi) for follow-up input DAG workflow directed acyclic graph adjust Degree calculates.
The workflow schedule method strengthening study based on the degree of depth the most according to claim 2, it is characterised in that sample pool tires out Meter refers to that the sample size of the DAG workflow directed acyclic graph accumulated in sample pool, more than 100, just starts to a certain extent Step C) calculate, training sample random can sample 100 DAG workflow directed acyclic graphs from sample pool and count Calculate.
4. according to the workflow schedule method strengthening study based on the degree of depth described in Claims 2 or 3, it is characterised in that step The training method DQN step of the neutral net released according to markov decision process in C) is as follows:
Step C1) for each time point t, there is markoff process et=(st,at,rt,st+1), define storage pool D=e1, e2…eN, initializing storage pool D size is N, initializes and is worth function of movement Q, and gradient descent method iterations is M time, and M is DAG Workflow directed acyclic graph number, initialization task state set stInThereinIt is s1Corresponding mapping function, thenIt is stCorresponding mapping function；
Step C2) after performing a period of time, obtain the scheduling scheme a of corresponding time point ttIf, scheduling scheme only one of which, For atItself, otherwiseObtain the task status set of task status set (t+1) time pointBy nowIt is stored in storage pool D；
Step C3) to make i be DAG workflow directed acyclic graph number in storage pool, if last DAG work that i is circulation Stream directed acyclic graph, then make yi=ri, riFor feedback function R (St,at) value in corresponding moment；If i is not last of circulation Open DAG workflow directed acyclic graph, then make riFor feedback function R (St,at) right Should the value in moment,It is stCorresponding mapping function, a ' is the scheduling scheme of corresponding time point t, and Υ is attenuation quotient；
Step C4) according to gradient descent algorithm, the loss function L to deep neural networki(θi)=ES, a～p ()[(yi-Q(s,a；θi))2], iteration M Secondary, i.e. this loss function is sought local derviation Try to achieve the minima of this loss function, and the minima of this loss function is substituted into formulaTo θiEnter Row updates, and wherein α is called learning rate is constant, until θiTill convergence or iteration complete, it is possible to obtain neutral net Parameter matrix θiValue.
The workflow schedule method strengthening study based on the degree of depth the most according to claim 1, it is characterised in that step B) in Every DAG workflow directed acyclic graph is carried out MDP markov decision process modeling procedure as follows:
Step B1) store DAG workflow directed acyclic graph with two-dimensional array G (T, E), wherein T represents the oriented nothing of DAG workflow Task node set in ring figure, E represents the limit collection in DAG workflow directed acyclic graph；
Step B2) definition (Ti,Tj) ∈ E represents that task j must perform to perform afterwards in task i, defines V (Ti,Ti) ＞ 0 table Show task TiThe execution time of estimating of itself, define (Vi,Tj) represent that if task i and task j are assigned to different nodes and hold The call duration time of row, as V (Ti, Tj) and there is no dependence between=∞ explanation task i and task j；
Step B3) by the two-dimensional matrix V of N × (N+1)N×(N+1)Represent distribution state S of whole workflow, wherein N Represent the sum of task node in DAG, the last string i.e. N+1 row of matrix, represent the distribution condition of each task；
Step B4) definition feedback functionWherein t (St) represent task status StUnder the time that always performs, C table Show original state S0Under the time that always performs,
CN201610656579.6A 2016-08-11 2016-08-11 The workflow schedule method of study is strengthened based on the degree of depth Pending CN106228314A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610656579.6A CN106228314A (en) 2016-08-11 2016-08-11 The workflow schedule method of study is strengthened based on the degree of depth

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610656579.6A CN106228314A (en) 2016-08-11 2016-08-11 The workflow schedule method of study is strengthened based on the degree of depth

Publications (1)

Publication Number Publication Date
CN106228314A true CN106228314A (en) 2016-12-14

Family

ID=57547189

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610656579.6A Pending CN106228314A (en) 2016-08-11 2016-08-11 The workflow schedule method of study is strengthened based on the degree of depth

Country Status (1)

Country Link
CN (1) CN106228314A (en)

Cited By (47)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106843225A (en) * 2017-03-15 2017-06-13 宜宾学院 A kind of Intelligent Mobile Robot path planning system
CN107145387A (en) * 2017-05-23 2017-09-08 南京大学 A kind of method for scheduling task learnt under vehicle-mounted net environment based on deeply
CN107315572A (en) * 2017-07-19 2017-11-03 北京上格云技术有限公司 Build control method, storage medium and the terminal device of Mechatronic Systems
CN107798388A (en) * 2017-11-23 2018-03-13 航天天绘科技有限公司 The method of TT＆C Resources dispatching distribution based on Multi Agent and DNN
CN108021028A (en) * 2017-12-22 2018-05-11 重庆邮电大学 A kind of various dimensions cooperative control method converted based on relevant redundancy with strengthening study
CN108197871A (en) * 2018-01-19 2018-06-22 顺丰科技有限公司 The mission planning method and system that express delivery receipts are dispatched officers
CN108282587A (en) * 2018-01-19 2018-07-13 重庆邮电大学 Mobile customer service dialogue management method under being oriented to strategy based on status tracking
CN108322541A (en) * 2018-02-09 2018-07-24 杭州顺网科技股份有限公司 A kind of adaptive Distributed architecture
CN108334439A (en) * 2018-03-14 2018-07-27 百度在线网络技术（北京）有限公司 A kind of method for testing pressure, device, equipment and storage medium
CN108494576A (en) * 2018-01-29 2018-09-04 中山大学 A kind of distributed parameters server updating method based on genetic algorithm
CN108596335A (en) * 2018-04-20 2018-09-28 浙江大学 A kind of adaptive crowdsourcing method based on deeply study
CN108897608A (en) * 2018-05-31 2018-11-27 中国科学院软件研究所 A kind of intelligent universal task scheduling system that data-driven is expansible
CN108960433A (en) * 2018-06-26 2018-12-07 第四范式（北京）技术有限公司 For running the method and system of machine learning modeling process
CN108964042A (en) * 2018-07-24 2018-12-07 合肥工业大学 Regional power grid operating point method for optimizing scheduling based on depth Q network
CN108958916A (en) * 2018-06-29 2018-12-07 杭州电子科技大学 Workflow unloads optimization algorithm under a kind of mobile peripheral surroundings
CN109101339A (en) * 2018-08-15 2018-12-28 北京邮电大学 Video task parallel method, device and Heterogeneous Cluster Environment in isomeric group
CN109709916A (en) * 2018-12-20 2019-05-03 宁波大学 A kind of dispatching method based on Gibbs sampling method
CN109754075A (en) * 2019-01-16 2019-05-14 中南民族大学 Dispatching method, equipment, storage medium and the device of wireless sensor network node
CN109815537A (en) * 2018-12-19 2019-05-28 清华大学 A kind of high-throughput material simulation calculation optimization method based on time prediction
CN110008002A (en) * 2019-04-09 2019-07-12 中国科学院上海高等研究院 Job scheduling method, device, terminal and medium based on Stationary Distribution probability
CN110020767A (en) * 2017-11-30 2019-07-16 西门子股份公司 Intervene the automatically coherent property inspection method after the workflow based on BPMN executes manually
CN110195660A (en) * 2019-06-19 2019-09-03 南京航空航天大学 Aero-engine control device based on depth Q study
CN110489223A (en) * 2019-08-26 2019-11-22 北京邮电大学 Method for scheduling task, device and electronic equipment in a kind of isomeric group
WO2020009139A1 (en) * 2018-07-04 2020-01-09 株式会社Ｐｒｅｆｅｒｒｅｄ Ｎｅｔｗｏｒｋｓ Learning method, learning device, learning system, and program
CN110809306A (en) * 2019-11-04 2020-02-18 电子科技大学 Terminal access selection method based on deep reinforcement learning
WO2020037156A1 (en) * 2018-08-16 2020-02-20 EMC IP Holding Company LLC Workflow optimization
CN110888401A (en) * 2018-09-11 2020-03-17 北京京东金融科技控股有限公司 Combustion control optimization method and device for thermal generator set and readable storage medium
CN111191934A (en) * 2019-12-31 2020-05-22 北京理工大学 Multi-target cloud workflow scheduling method based on reinforcement learning strategy
CN111343651A (en) * 2020-02-18 2020-06-26 电子科技大学 Service chain deployment method and system for serving crowd-sourcing computing environment
CN111445081A (en) * 2020-04-01 2020-07-24 浙江大学 Digital twin virtual-real self-adaptive iterative optimization method for dynamic scheduling of product operation
CN111465031A (en) * 2020-03-26 2020-07-28 南京理工大学 Dynamic node scheduling method based on DQN algorithm in wireless body area network
CN111510319A (en) * 2020-03-06 2020-08-07 重庆邮电大学 Network slice resource management method based on state perception
CN111506405A (en) * 2020-04-08 2020-08-07 北京交通大学 Edge calculation time slice scheduling method based on deep reinforcement learning
CN111525587A (en) * 2020-04-01 2020-08-11 中国电力科学研究院有限公司 Reactive load situation-based power grid reactive voltage control method and system
CN111756653A (en) * 2020-06-04 2020-10-09 北京理工大学 Multi-coflow scheduling method based on deep reinforcement learning of graph neural network
CN111812519A (en) * 2020-07-15 2020-10-23 南京航空航天大学 Battery parameter identification method and system
CN112204580A (en) * 2018-03-27 2021-01-08 诺基亚通信公司 Method and apparatus for facilitating resource pairing using deep Q networks
CN112256961A (en) * 2020-10-19 2021-01-22 平安科技（深圳）有限公司 User portrait generation method, device, equipment and medium
CN112685165A (en) * 2021-01-08 2021-04-20 北京理工大学 Multi-target cloud workflow scheduling method based on joint reinforcement learning strategy
CN112809678A (en) * 2021-01-15 2021-05-18 合肥工业大学 Cooperative control method for production line system of multi-robot workstation
CN113033928A (en) * 2019-12-09 2021-06-25 南京行者易智能交通科技有限公司 Design method, device and system of bus shift scheduling model based on deep reinforcement learning
CN113487165A (en) * 2021-07-01 2021-10-08 福州大学 Intelligent factory production operation scheduling method and system based on deep reinforcement learning
CN113824650A (en) * 2021-08-13 2021-12-21 上海光华智创网络科技有限公司 Parameter transmission scheduling algorithm and system in distributed deep learning system
CN113888136A (en) * 2021-10-21 2022-01-04 北京航空航天大学 Workflow scheduling method based on DQN algorithm principle
CN114545884A (en) * 2022-03-16 2022-05-27 温州大学 Equivalent parallel machine dynamic intelligent scheduling method based on enhanced topological neural evolution
CN114860398A (en) * 2022-04-21 2022-08-05 郑州大学 Task scheduling method, device and equipment of intelligent cloud platform
WO2023241000A1 (en) * 2022-06-15 2023-12-21 苏州元脑智能科技有限公司 Dag task scheduling method and apparatus, device, and storage medium

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103226759A (en) * 2013-04-25 2013-07-31 中山大学 Dynamic cloud workflow scheduling method based on genetic algorithm
CN103412792A (en) * 2013-07-18 2013-11-27 成都国科海博计算机系统有限公司 Dynamic task scheduling method and device under cloud computing platform environment
CN104657221A (en) * 2015-03-12 2015-05-27 广东石油化工学院 Multi-queue peak-alternation scheduling model and multi-queue peak-alteration scheduling method based on task classification in cloud computing

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103226759A (en) * 2013-04-25 2013-07-31 中山大学 Dynamic cloud workflow scheduling method based on genetic algorithm
CN103412792A (en) * 2013-07-18 2013-11-27 成都国科海博计算机系统有限公司 Dynamic task scheduling method and device under cloud computing platform environment
CN104657221A (en) * 2015-03-12 2015-05-27 广东石油化工学院 Multi-queue peak-alternation scheduling model and multi-queue peak-alteration scheduling method based on task classification in cloud computing

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
VOLODYMYR MNIH等: "《Playing Atari with Deep Reinforcement Learning》", 19 December 2013 *
陈圣磊，等: "协同设计任务调度的多步Q学习算法", 《计算机辅助设计与图形学学报》 *

Cited By (81)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106843225A (en) * 2017-03-15 2017-06-13 宜宾学院 A kind of Intelligent Mobile Robot path planning system
CN107145387A (en) * 2017-05-23 2017-09-08 南京大学 A kind of method for scheduling task learnt under vehicle-mounted net environment based on deeply
CN107145387B (en) * 2017-05-23 2019-09-10 南京大学 A kind of method for scheduling task based on deeply study under vehicle-mounted net environment
CN107315572B (en) * 2017-07-19 2020-08-11 北京上格云技术有限公司 Control method of building electromechanical system, storage medium and terminal equipment
CN107315572A (en) * 2017-07-19 2017-11-03 北京上格云技术有限公司 Build control method, storage medium and the terminal device of Mechatronic Systems
CN107798388A (en) * 2017-11-23 2018-03-13 航天天绘科技有限公司 The method of TT＆C Resources dispatching distribution based on Multi Agent and DNN
CN107798388B (en) * 2017-11-23 2022-02-08 航天天绘科技有限公司 Measurement and control resource scheduling and allocation method based on Multi-Agent and DNN
CN110020767A (en) * 2017-11-30 2019-07-16 西门子股份公司 Intervene the automatically coherent property inspection method after the workflow based on BPMN executes manually
CN108021028A (en) * 2017-12-22 2018-05-11 重庆邮电大学 A kind of various dimensions cooperative control method converted based on relevant redundancy with strengthening study
CN108021028B (en) * 2017-12-22 2019-04-09 重庆邮电大学 It is a kind of to be converted based on relevant redundancy and enhance the various dimensions cooperative control method learnt
CN108197871A (en) * 2018-01-19 2018-06-22 顺丰科技有限公司 The mission planning method and system that express delivery receipts are dispatched officers
CN108282587B (en) * 2018-01-19 2020-05-26 重庆邮电大学 Mobile customer service conversation management method based on state tracking and policy guidance
CN108282587A (en) * 2018-01-19 2018-07-13 重庆邮电大学 Mobile customer service dialogue management method under being oriented to strategy based on status tracking
CN108494576A (en) * 2018-01-29 2018-09-04 中山大学 A kind of distributed parameters server updating method based on genetic algorithm
CN108322541B (en) * 2018-02-09 2021-04-06 杭州顺网科技股份有限公司 Self-adaptive distributed system architecture
CN108322541A (en) * 2018-02-09 2018-07-24 杭州顺网科技股份有限公司 A kind of adaptive Distributed architecture
CN108334439B (en) * 2018-03-14 2021-06-04 百度在线网络技术（北京）有限公司 Pressure testing method, device, equipment and storage medium
CN108334439A (en) * 2018-03-14 2018-07-27 百度在线网络技术（北京）有限公司 A kind of method for testing pressure, device, equipment and storage medium
CN112204580A (en) * 2018-03-27 2021-01-08 诺基亚通信公司 Method and apparatus for facilitating resource pairing using deep Q networks
CN112204580B (en) * 2018-03-27 2024-04-12 诺基亚通信公司 Method and apparatus for facilitating resource pairing using deep Q networks
CN108596335B (en) * 2018-04-20 2020-04-17 浙江大学 Self-adaptive crowdsourcing method based on deep reinforcement learning
CN108596335A (en) * 2018-04-20 2018-09-28 浙江大学 A kind of adaptive crowdsourcing method based on deeply study
CN108897608B (en) * 2018-05-31 2021-09-07 中国科学院软件研究所 Data-driven extensible intelligent general task scheduling system
CN108897608A (en) * 2018-05-31 2018-11-27 中国科学院软件研究所 A kind of intelligent universal task scheduling system that data-driven is expansible
CN108960433A (en) * 2018-06-26 2018-12-07 第四范式（北京）技术有限公司 For running the method and system of machine learning modeling process
CN108960433B (en) * 2018-06-26 2022-04-05 第四范式（北京）技术有限公司 Method and system for running machine learning modeling process
CN108958916B (en) * 2018-06-29 2021-06-22 杭州电子科技大学 Workflow unloading optimization method under mobile edge environment
CN108958916A (en) * 2018-06-29 2018-12-07 杭州电子科技大学 Workflow unloads optimization algorithm under a kind of mobile peripheral surroundings
WO2020009139A1 (en) * 2018-07-04 2020-01-09 株式会社Ｐｒｅｆｅｒｒｅｄ Ｎｅｔｗｏｒｋｓ Learning method, learning device, learning system, and program
JPWO2020009139A1 (en) * 2018-07-04 2021-07-08 株式会社Ｐｒｅｆｅｒｒｅｄ Ｎｅｔｗｏｒｋｓ Robot control devices, systems, robot control methods, policy update methods, and neural networks
JP7398373B2 (en) 2018-07-04 2023-12-14 株式会社Preferred Networks Control device, system, control method, and program
CN108964042A (en) * 2018-07-24 2018-12-07 合肥工业大学 Regional power grid operating point method for optimizing scheduling based on depth Q network
CN108964042B (en) * 2018-07-24 2021-10-15 合肥工业大学 Regional power grid operating point scheduling optimization method based on deep Q network
CN109101339A (en) * 2018-08-15 2018-12-28 北京邮电大学 Video task parallel method, device and Heterogeneous Cluster Environment in isomeric group
CN109101339B (en) * 2018-08-15 2019-05-31 北京邮电大学 Video task parallel method, device and Heterogeneous Cluster Environment in isomeric group
WO2020037156A1 (en) * 2018-08-16 2020-02-20 EMC IP Holding Company LLC Workflow optimization
US11868890B2 (en) 2018-08-16 2024-01-09 Landmark Graphics Corporation Workflow optimization
GB2587979A (en) * 2018-08-16 2021-04-14 Landmark Graphics Corp Workflow optimization
US11315014B2 (en) 2018-08-16 2022-04-26 EMC IP Holding Company LLC Workflow optimization
CN110888401A (en) * 2018-09-11 2020-03-17 北京京东金融科技控股有限公司 Combustion control optimization method and device for thermal generator set and readable storage medium
CN109815537A (en) * 2018-12-19 2019-05-28 清华大学 A kind of high-throughput material simulation calculation optimization method based on time prediction
CN109709916A (en) * 2018-12-20 2019-05-03 宁波大学 A kind of dispatching method based on Gibbs sampling method
CN109754075A (en) * 2019-01-16 2019-05-14 中南民族大学 Dispatching method, equipment, storage medium and the device of wireless sensor network node
CN110008002A (en) * 2019-04-09 2019-07-12 中国科学院上海高等研究院 Job scheduling method, device, terminal and medium based on Stationary Distribution probability
CN110008002B (en) * 2019-04-09 2022-11-29 中国科学院上海高等研究院 Job scheduling method, device, terminal and medium based on stable distribution probability
CN110195660B (en) * 2019-06-19 2020-04-21 南京航空航天大学 Aero-engine control device based on deep Q learning
CN110195660A (en) * 2019-06-19 2019-09-03 南京航空航天大学 Aero-engine control device based on depth Q study
CN110489223A (en) * 2019-08-26 2019-11-22 北京邮电大学 Method for scheduling task, device and electronic equipment in a kind of isomeric group
CN110489223B (en) * 2019-08-26 2022-03-29 北京邮电大学 Task scheduling method and device in heterogeneous cluster and electronic equipment
CN110809306A (en) * 2019-11-04 2020-02-18 电子科技大学 Terminal access selection method based on deep reinforcement learning
CN113033928A (en) * 2019-12-09 2021-06-25 南京行者易智能交通科技有限公司 Design method, device and system of bus shift scheduling model based on deep reinforcement learning
CN113033928B (en) * 2019-12-09 2023-10-31 南京行者易智能交通科技有限公司 Method, device and system for designing bus shift model based on deep reinforcement learning
CN111191934A (en) * 2019-12-31 2020-05-22 北京理工大学 Multi-target cloud workflow scheduling method based on reinforcement learning strategy
CN111191934B (en) * 2019-12-31 2022-04-15 北京理工大学 Multi-target cloud workflow scheduling method based on reinforcement learning strategy
CN111343651B (en) * 2020-02-18 2021-11-16 电子科技大学 Service chain deployment method and system for serving crowd-sourcing computing environment
CN111343651A (en) * 2020-02-18 2020-06-26 电子科技大学 Service chain deployment method and system for serving crowd-sourcing computing environment
CN111510319B (en) * 2020-03-06 2022-07-08 重庆邮电大学 Network slice resource management method based on state perception
CN111510319A (en) * 2020-03-06 2020-08-07 重庆邮电大学 Network slice resource management method based on state perception
CN111465031B (en) * 2020-03-26 2022-10-14 南京理工大学 Dynamic node scheduling method based on DQN algorithm in wireless body area network
CN111465031A (en) * 2020-03-26 2020-07-28 南京理工大学 Dynamic node scheduling method based on DQN algorithm in wireless body area network
CN111525587B (en) * 2020-04-01 2022-10-25 中国电力科学研究院有限公司 Reactive load situation-based power grid reactive voltage control method and system
CN111445081A (en) * 2020-04-01 2020-07-24 浙江大学 Digital twin virtual-real self-adaptive iterative optimization method for dynamic scheduling of product operation
CN111525587A (en) * 2020-04-01 2020-08-11 中国电力科学研究院有限公司 Reactive load situation-based power grid reactive voltage control method and system
CN111506405A (en) * 2020-04-08 2020-08-07 北京交通大学 Edge calculation time slice scheduling method based on deep reinforcement learning
CN111756653A (en) * 2020-06-04 2020-10-09 北京理工大学 Multi-coflow scheduling method based on deep reinforcement learning of graph neural network
CN111812519A (en) * 2020-07-15 2020-10-23 南京航空航天大学 Battery parameter identification method and system
CN112256961A (en) * 2020-10-19 2021-01-22 平安科技（深圳）有限公司 User portrait generation method, device, equipment and medium
CN112256961B (en) * 2020-10-19 2024-04-09 平安科技（深圳）有限公司 User portrait generation method, device, equipment and medium
CN112685165B (en) * 2021-01-08 2022-08-23 北京理工大学 Multi-target cloud workflow scheduling method based on joint reinforcement learning strategy
CN112685165A (en) * 2021-01-08 2021-04-20 北京理工大学 Multi-target cloud workflow scheduling method based on joint reinforcement learning strategy
CN112809678A (en) * 2021-01-15 2021-05-18 合肥工业大学 Cooperative control method for production line system of multi-robot workstation
CN113487165B (en) * 2021-07-01 2024-05-03 福州大学 Intelligent factory production job scheduling method and system based on deep reinforcement learning
CN113487165A (en) * 2021-07-01 2021-10-08 福州大学 Intelligent factory production operation scheduling method and system based on deep reinforcement learning
CN113824650B (en) * 2021-08-13 2023-10-20 上海光华智创网络科技有限公司 Parameter transmission scheduling algorithm and system in distributed deep learning system
CN113824650A (en) * 2021-08-13 2021-12-21 上海光华智创网络科技有限公司 Parameter transmission scheduling algorithm and system in distributed deep learning system
CN113888136A (en) * 2021-10-21 2022-01-04 北京航空航天大学 Workflow scheduling method based on DQN algorithm principle
CN114545884B (en) * 2022-03-16 2023-12-05 温州大学 Equivalent parallel machine dynamic intelligent scheduling method based on enhanced topological neural evolution
CN114545884A (en) * 2022-03-16 2022-05-27 温州大学 Equivalent parallel machine dynamic intelligent scheduling method based on enhanced topological neural evolution
CN114860398A (en) * 2022-04-21 2022-08-05 郑州大学 Task scheduling method, device and equipment of intelligent cloud platform
CN114860398B (en) * 2022-04-21 2024-09-06 郑州大学 Intelligent cloud platform task scheduling method, device and equipment
WO2023241000A1 (en) * 2022-06-15 2023-12-21 苏州元脑智能科技有限公司 Dag task scheduling method and apparatus, device, and storage medium

Similar Documents

Publication Publication Date Title
CN106228314A (en) The workflow schedule method of study is strengthened based on the degree of depth
CN110489223B (en) Task scheduling method and device in heterogeneous cluster and electronic equipment
Wang et al. Learning scheduling policies for multi-robot coordination with graph attention networks
Caldeira et al. A Pareto based discrete Jaya algorithm for multi-objective flexible job shop scheduling problem
Baer et al. Multi-agent reinforcement learning for job shop scheduling in flexible manufacturing systems
Fattahi et al. Dynamic scheduling in flexible job shop systems by considering simultaneously efficiency and stability
Li et al. An effective shuffled frog-leaping algorithm for multi-objective flexible job shop scheduling problems
CN111756653B (en) Multi-coflow scheduling method based on deep reinforcement learning of graph neural network
CN109388484A (en) A kind of more resource cloud job scheduling methods based on Deep Q-network algorithm
CN104408518B (en) Based on the neural network learning optimization method of particle swarm optimization algorithm
CN101520858B (en) Ant colony optimization-differential evolution fusion method for solving traveling salesman problems
Nilakantan et al. Design of energy efficient RAL system using evolutionary algorithms
CN111325356A (en) Neural network search distributed training system and training method based on evolutionary computation
Brajevic Artificial bee colony algorithm for the capacitated vehicle routing problem
CN112711475B (en) Workflow scheduling method and system based on graph convolution neural network
Ghassemi et al. Decentralized dynamic task allocation in swarm robotic systems for disaster response
Li et al. Colored traveling salesman problem and solution
Mondal et al. A genetic algorithm-based approach to solve a new time-limited travelling salesman problem
Zhang et al. Modelling and simulation of the task scheduling behavior in collaborative product development process
Gonzalez-Pardo et al. A new CSP graph-based representation for ant colony optimization
Cao et al. An adaptive multi-strategy artificial bee colony algorithm for integrated process planning and scheduling
Chica et al. A new diversity induction mechanism for a multi-objective ant colony algorithm to solve a real-world time and space assembly line balancing problem
El-Ghamrawy et al. An agent decision support module based on granular rough model
Gao et al. Collaborative scheduling with adaptation to failure for heterogeneous robot teams
NADER et al. A multi-mode resource-constrained optimization of time-cost trade-off problems in project scheduling using a genetic algorithm

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20161214

RJ01 Rejection of invention patent application after publication