CN112905312A - Workflow scheduling method based on deep Q neural network in edge computing environment - Google Patents

Workflow scheduling method based on deep Q neural network in edge computing environment Download PDF

Info

Publication number
CN112905312A
CN112905312A CN202110074556.5A CN202110074556A CN112905312A CN 112905312 A CN112905312 A CN 112905312A CN 202110074556 A CN202110074556 A CN 202110074556A CN 112905312 A CN112905312 A CN 112905312A
Authority
CN
China
Prior art keywords
task
workflow
time
edge
node
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
CN202110074556.5A
Other languages
Chinese (zh)
Inventor
俞东进
项媛媛
黄彬彬
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Dianzi University
Original Assignee
Hangzhou Dianzi University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Dianzi University filed Critical Hangzhou Dianzi University
Priority to CN202110074556.5A priority Critical patent/CN112905312A/en
Publication of CN112905312A publication Critical patent/CN112905312A/en
Withdrawn legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Abstract

The invention mainly discloses a workflow scheduling method SAWS based on deep reinforcement learning in an edge computing environment. The main process of the implementation of the invention is to construct the problem into a Markov decision process, define the reward, the state and the action corresponding to the workflow scheduling problem in the edge computing environment, then calculate the weight of the task nodes according to the execution duration, the transmission duration and the dependence condition of the task nodes in the workflow for sequencing, and then make a decision on the scheduling of the task nodes based on the deep Q network. The main objective of the SAWS strategy is to find a task scheduling strategy which can minimize the long-term execution delay of the workflow on the premise of ensuring the safety of user information. The invention greatly improves the execution efficiency of the workflow in the mobile edge network environment and ensures the information safety of the user through the learning and decision of the Q neural network.

Description

Workflow scheduling method based on deep Q neural network in edge computing environment
Technical Field
The invention mainly relates to the field of deep reinforcement learning and edge computing, in particular to a workflow scheduling method based on a deep Q neural network in an edge computing environment.
Background
Mobile Edge Computing (MEC) based mobile edge networks can provide low latency and high computational load for popular mobile applications (e.g., virtual/augmented reality, mobile games, in-vehicle network applications, etc.). In a mobile edge network, an edge cloud server with computing and storage functions is deployed near a mobile user, and for a mobile device, offloading service onto the edge server can provide the best quality of service for the mobile user, i.e., minimal response delay.
For mobile applications it may be defined as the execution of a series of tasks, and the order of execution of these tasks results from the dependency between the tasks on the result data. A workflow generated by a mobile device can be generally represented by a Directed Acyclic Graph (DAG), where a set of task nodes in the workflow correspond to a set of nodes in the graph, and relationships in which data dependencies exist between task nodes are represented as directed edges in the graph. In addition to the difficulties presented by scheduling workflow tasks, there are two difficulties: (1) moving edges compute the dynamic ignorance of the scene. (2) The information interaction between the user and the edge server has the possibility of data leakage and data tampering, and the user is lost.
Therefore, how to guarantee the service quality and information security of the mobile user workflow scheduling in the mobile edge computing environment is an important issue in the mobile edge computing research.
Disclosure of Invention
In order to solve the above problems, the present invention provides a workflow scheduling method based on a deep Q neural network in an edge computing environment.
The invention comprises the following steps:
s1, constructing an edge computing environment model:
mobile device is denoted by U, set eNB ═ eNB1,…,eNBi,…,eNBnRepresents n edge servers;
computing power of mobile user is represented by CuRepresenting each edge server to computing power by CiRepresents;
the transmission rate between the mobile device U and the n edge servers is
Figure BDA0002907041950000011
Denotes eNB at the t-th time sliceiAnd U.
S2, generating a workflow:
setting the number of task nodes contained in the workflow generated by the mobile equipment as K, randomly arranging 1-K, and generating a corresponding directed acyclic graph G as a topological sorting result of the directed acyclic graph according to the arrangement sequence<V,E>As a workflow generated by the mobile device U; set of nodes in directed acyclic graph, V ═ V1,…,vk,…,vKAs the set of task nodes in the workflow, the set of directed edges E ═ E in the directed acyclic graphkl|vk∈V,vlE is V and is used as a set of dependency relationships among task nodes in the workflow;
directed edge eklIndicating that only task v is being executedkThe edge server of (2) will task vkIs sent to the ready to execute task vlOn the edge server, task vlCan be executed; and there is only one start node v in the workflowstartAs the task node executed first in the workflow, there is also only one end node vendAnd the time for completing the execution of the end node is the time for completing the execution of the workflow.
S3, task node priority ordering:
for each task node vkAssigning a weight Pr (v)k) To realize the sequencing of the task node priority; weight Pr (v)k) The calculation is as follows:
Figure BDA0002907041950000021
wherein
Figure BDA0002907041950000022
Denotes vkAverage time performed on all edge servers; rcIs a fixed constant and represents the transmission rate of data between all edge servers, succ (v)k) Representing task nodes vkAll of the nodes of the predecessor are,
Figure BDA0002907041950000023
a data size representing a calculation result;
after the weights of the task nodes in all the workflows are obtained by calculating step by step from the end node to the precursor nodes, the weights are sorted in a descending order according to the weight, and the sorted weights are used as the execution order of the task nodes in the workflows.
S4, risk constraint:
task node v at mobile devicekThe probability of the task data being leaked or modified during the data transmission to the ith edge server is
Figure BDA0002907041950000024
Figure BDA0002907041950000025
The risk probability of the task data being attacked is
Figure BDA0002907041950000026
And its risk probability P (v)k) Probability of risk P that must be less than or equal to the scene settingmax。
S5, constructing a Markov decision process model in the mobile edge environment, which comprises the following steps:
s51, defining the state of the system
Figure BDA0002907041950000027
Wherein Wc(τ)＝{Wc,1(τ),…,Wc,i(τ),,…,Wc,n(τ) } represents a set of task nodes offloaded to the corresponding edge server;
Figure BDA0002907041950000028
representing a transmission rate between the mobile device and the corresponding edge server;
s52, defining the action of the system
Figure BDA0002907041950000031
Wherein a isc(τ)＝{ac,1(τ),…,ac,i(τ),,…,ac,n(τ) } indicates that the task nodes in the workflow are unloaded to a certain edge server;
Figure BDA0002907041950000032
indicating the security level of the cryptographic service selected by the mobile device when scheduling a task node in the workflow,
Figure BDA0002907041950000033
representing the security level of the guaranteed data integrity service selected by the mobile device when scheduling the task node in the workflow;
s53, defining reward (tau) of the system as-Tend(vk) Wherein T isend(vk) Representing a node vkThe latest completion time of (c); latest time of completion
Figure BDA0002907041950000034
Figure BDA0002907041950000035
Comprises a task node vkStart time T ofstart(vk) Time of encryption
Figure BDA0002907041950000036
Time of flight
Figure BDA0002907041950000037
Waiting time
Figure BDA0002907041950000038
Decryption time
Figure BDA0002907041950000039
And execution time
Figure BDA00029070419500000310
S6, building a depth Q network:
the deep Q neural network comprises an estimation Q neural network, a target neural network and an experience pool;
the Q neural network and the target Q neural network have the same network structure, and the Q neural network can periodically transmit network parameters to the target neural network;
the experience pool is used for storing state transition samples obtained by interaction with the environment in each time slice, and the quaternion group with fixed batch is randomly extracted from the experience pool to train the estimation Q neural network in each learning.
S7, algorithm implementation:
a constant value epsilon is given as learning time, and all task nodes in the workflow are considered to be completed by one-time learning after learning and scheduling; in the workflow scheduling process, setting the current time slice as 0; at the beginning of the tau-th time slice, selecting and executing an action a (tau) by observing the current state s (tau) of the moving edge environment, calculating a reward R (s (tau), a (tau)) after executing the action and observing the state s (tau' +1) of the system after executing the action, and storing the reward R (s (tau), a (tau)) and the state s into an experience pool; when enough data is stored in the experience pool, sampling learning is started.
The invention greatly improves the execution efficiency of the workflow in the mobile edge network environment and ensures the information safety of the user through the learning and decision of the Q neural network.
Drawings
FIG. 1 is a flowchart of a workflow task scheduling method based on deep reinforcement learning according to the present invention;
FIG. 2 is an architecture diagram of workflow scheduling in a mobile edge computing environment;
FIG. 3 is a diagram of a workflow scheduling strategy based on deep reinforcement learning security awareness;
FIG. 4 is a workflow convergence diagram with a task node number of 100;
FIG. 5 is a comparison of the inventive algorithm and the AWM algorithm for risk probability changes;
FIG. 6 is a comparison of the algorithm of the present invention and the AWM algorithm for varying server computing power;
fig. 7 is a comparison of the inventive algorithm and the AWM algorithm with a change in the number of servers.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to the accompanying drawings.
On the contrary, the invention is intended to cover alternatives, modifications, equivalents and alternatives which may be included within the spirit and scope of the invention as defined by the appended claims. Furthermore, in the following detailed description of the present invention, certain specific details are set forth in order to provide a better understanding of the present invention. It will be apparent to one skilled in the art that the present invention may be practiced without these specific details.
As shown in FIG. 1, the workflow scheduling method based on the deep Q neural network in the edge computing environment of the invention includes the following steps:
s1, constructing an edge computing environment model: in a practical scenario, a mobile device is served by a plurality of edge servers around it. A mobile device is denoted by U in the context of the present invention, by the set eNB ═ eNB1,…,eNBi,…,eNBnRepresents n edge servers. In addition, only one group of workflows generated by the mobile equipment is learned and scheduled. The mobile device only needs to encrypt the task node data and then transmit the task data to the edge server through the wireless network, and the task data decryption and task execution work are carried out on the edge server. Computing power of mobile user is represented by CuRepresenting each edge server to computing power by CiAnd (4) showing. The transmission rate between the mobile device and the n edge servers is changed due to the mobility of the mobile device, so
Figure BDA0002907041950000041
Denotes eNB at the t-th time sliceiAnd U.
S2, generating a workflow: setting the number of task nodes contained in the workflow generated by the mobile equipment as K, randomly arranging 1-K, and arranging the task nodes according to the arrangementThe column order is used as the topological sorting result of the directed acyclic graph to generate a corresponding directed acyclic graph G ═ G-<V,E>As a workflow generated by the mobile device U, the set of nodes in the graph, V ═ V1,…,vk,…,vKAs the collection of task nodes in the workflow, the directed edge collection E ═ E in the graphkl|vk∈V,vlE.v as a set of dependencies between task nodes in the workflow. Any task node v in the workflow is different in the executed task contentkThe working load W thereofkData size
Figure BDA0002907041950000042
And the data size of the calculation result
Figure BDA0002907041950000043
Or may be different from other task nodes in the workflow. Directed edge eklIndicating that only task v is being executedkThe edge server of (2) will task vkIs sent to the ready to execute task vlOn the edge server, task vlCan it be executed. And there is only one start node v in the workflowstartAs the task node executed first in the workflow, there is also only one end node vendAnd the time for completing the execution of the end node is the time for completing the execution of the workflow.
S3, task node priority ordering: in the invention, each task node v is divided into a plurality of task nodeskAssigning a weight Pr (v)k) To achieve the ordering of task node priorities. Weight Pr (v)k) Can be calculated by
Figure BDA0002907041950000051
Figure BDA0002907041950000052
Thus obtaining the product. Since the task nodes have not been scheduled in the sequencing phase, the task node vkIt is not known to which edge server it will be offloaded to execute, so v is taken into account in the weight calculationkIn all edgesAverage time of execution on a server
Figure BDA0002907041950000053
RcThe rate of data transmission between all edge servers is represented by a fixed constant value and calculated
Figure BDA0002907041950000054
The time at which the mobile device delivers the task data to the edge server is available. Since the execution sequence of the task nodes in the workflow has a dependency relationship, the task nodes v are calculatedkThe weight of (c) is taken into account for all its predecessor nodes succ (v)k) The weight of (c). After the weights of the task nodes in all the workflows are obtained by calculating step by step from the end node to the precursor nodes, the weights are sorted in a descending order according to the weight, and the sorted weights are used as the execution order of the task nodes in the workflows.
S4, risk constraint: task node v at mobile devicekThe probability of the task data being leaked or modified during the data transmission to the ith edge server is
Figure BDA0002907041950000055
Figure BDA0002907041950000056
So the risk probability of the task data being attacked is
Figure BDA0002907041950000057
Figure BDA0002907041950000058
And its risk probability P (v)k) Probability of risk P that must be less than or equal to the scene settingmax。
S5, constructing a Markov decision process model in the mobile edge environment, and specifically comprising the following steps:
s51, defining the state of the system
Figure BDA0002907041950000059
Wherein Wc(τ)＝{Wc,1(τ),…,Wc,i(τ),,…,Wc,n(τ) } represents a set of task nodes offloaded to the corresponding edge server;
Figure BDA00029070419500000510
representing the transmission rate between the mobile device and the corresponding edge server.
S52, defining the action of the system
Figure BDA00029070419500000511
Wherein a isc(τ)＝{ac,1(τ),…,ac,i(τ),,…,ac,n(τ) } indicates to which edge server the task node in the workflow is offloaded.
Figure BDA00029070419500000512
Indicating the security level of the cryptographic service selected by the mobile device when scheduling a task node in the workflow,
Figure BDA00029070419500000513
indicating the security level selected by the mobile device to guarantee data integrity services when scheduling task nodes in the workflow.
S53, defining reward (tau) of the system as-Tend(vk) Wherein T isend(vk) Representing a node vkThe latest completion time of (c). And the latest completion time
Figure BDA0002907041950000061
Figure BDA0002907041950000062
Requiring the calculation of task nodes v firstkStart time T ofstart(vk) Time of encryption
Figure BDA0002907041950000063
Time of flight
Figure BDA0002907041950000064
Waiting time
Figure BDA0002907041950000065
Decryption time
Figure BDA0002907041950000066
And execution time
Figure BDA0002907041950000067
Wherein the start time
Figure BDA0002907041950000068
pre(vk) Is v iskOf predecessor nodes, Tend(vh) Is any predecessor node vhThe latest time of completion of the process,
Figure BDA0002907041950000069
is the edge server executing vh for any predecessor node will vhIs transmitted to the executing node v which is currently scheduledkBy the edge server, which can
Figure BDA00029070419500000610
And (4) calculating. Additionally encrypting time
Figure BDA00029070419500000611
Figure BDA00029070419500000612
Decryption time
Figure BDA00029070419500000613
Waiting time
Figure BDA00029070419500000614
The time for executing all task nodes in the server queue on the ith edge server is obtained
Figure BDA00029070419500000615
Time of flight
Figure BDA00029070419500000616
S6, building a depth Q network: the deep Q neural network is mainly constructed by three functional components. The method comprises the following steps: an estimated Q neural network, a target neural network, and an experience pool. The Q neural network has the same network structure as the target Q neural network, and the Q neural network can periodically transmit the network parameters to the target neural network. The experience pool is used for storing state transition samples obtained by interaction with the environment in each time slice, and the quaternion group with fixed batch is randomly extracted from the experience pool to train the estimation Q neural network in each learning.
S7, algorithm implementation: and giving a constant value epsilon as learning time, and finishing learning and scheduling all task nodes in the workflow to be regarded as finishing learning once. In the workflow scheduling process, the current time slice is set to 0. At the beginning of the τ -th time slice, action a (τ) is selected and executed by observing the current state s (τ) of the moving edge environment, the reward R (s (τ), a (τ)) after the action is executed and the state s (τ' +1) of the system after the action is executed are calculated and stored in the experience pool. When enough data is stored in the experience pool, sampling learning is started.
The invention also realizes an AWM baseline algorithm, and the task nodes in the workflow are scheduled to the edge server with the minimum load for execution, compared with the SAWS strategy of the invention. And the influence of the risk probability change, the server computing capacity change and the server quantity change on the workflow execution time is evaluated respectively. Also from a comparison of the above scenarios, it can be observed that the SAWS strategy is superior to the AWM strategy.
Taking the workflow with the task node number of 100 as an example, the learning situation of the SAWS policy is shown in fig. 4, and it can be observed that the more the learning times are, the shorter the execution time of the workflow is and the later stage of learning gradually becomes stable, which shows that the execution delay of the workflow in the mobile edge environment can be effectively reduced by the present invention.
Risk probability: sequentially executing the risk probability P of the tasks in the workflowmaxAre respectively set to 0.2, 0.4, 0.6 and 0.81.0 the experiment was carried out. The workflow execution of the two strategies at different risk probabilities is shown in fig. 5. It can be observed from the figure that the higher the risk probability, the shorter the time to complete the workflow.
Edge server computing power: experiments were performed with the computing power of all edge servers set to 15GHz/s, 17.5GHz/s, 20GHz/s, 22.5GHz/s, 25GHz/s, respectively. Workflow execution for both policies when the edge servers are not of equal computing power is shown in fig. 6. It can be observed from the figure that the more computing power at the edge server, the shorter the time to complete the workflow. This is mainly because the stronger the computing power of the edge server, the shorter the time to decrypt and execute a task, and the shorter the waiting time for a task when other tasks are offloaded to the edge server.
Number of edge servers: experiments were performed with the number of edge servers in the environment changed in sequence to 2, 4, 6, 8, 10. Fig. 7 shows the workflow execution of two strategies when the number of edge servers is different. It can be observed that as the number of edge servers increases, the time to complete the workflow is shorter. The main reason is that the number of edge servers in the environment is increased, the number of task nodes unloaded to each edge server is correspondingly reduced, and the waiting time for task execution is reduced.

Claims (5)

1. The workflow scheduling method based on the deep Q neural network in the edge computing environment is characterized by comprising the following steps:
s1, constructing an edge computing environment model:
mobile device is denoted by U, set eNB ═ eNB1，...，eNBi，...，eNBnRepresents n edge servers;
computing power of mobile user is represented by CuRepresenting each edge server to computing power by CiRepresents;
the transmission rate between the mobile device U and the n edge servers is
Figure FDA0002907041940000011
Denotes eNB at the t-th time sliceiAnd the transmission rate between U;
s2, generating a workflow:
setting the number of task nodes contained in the workflow generated by the mobile equipment as K, randomly arranging 1-K, and generating a corresponding directed acyclic graph G as a topological sorting result of the directed acyclic graph according to the arrangement sequence<V，E>As a workflow generated by the mobile device U; set of nodes in directed acyclic graph, V ═ V1，...，vk，...，vKAs the set of task nodes in the workflow, the set of directed edges E ═ E in the directed acyclic graphkl|vk∈V，vlE is V and is used as a set of dependency relationships among task nodes in the workflow;
directed edge eklIndicating that only task v is being executedkThe edge server of (2) will task vkIs sent to the ready to execute task vlOn the edge server, task vlCan be executed; and there is only one start node v in the workflowstartAs the task node executed first in the workflow, there is also only one end node vendAnd the time for finishing the execution of the node is the time for finishing the execution of the workflow;
s3, task node priority ordering:
for each task node vkAssigning a weight Pr (v)k) To realize the sequencing of the task node priority; weight Pr (v)k) The calculation is as follows:
Figure FDA0002907041940000012
wherein
Figure FDA0002907041940000013
Denotes vkAverage time performed on all edge servers; rcIs a fixed constant and represents the mutual transmission between all edge serversRate of data, succ (v)k) Representing task nodes vkAll of the nodes of the predecessor are,
Figure FDA0002907041940000014
a data size representing a calculation result;
after the weights of the task nodes in all the workflows are obtained through step-by-step calculation from the end node to the precursor nodes, the weights are sorted in a descending order according to the weight, and the sorted weights are used as the execution order of the task nodes in the workflows;
s4, risk constraint:
task node v at mobile devicekThe probability of the task data being leaked or modified during the data transmission to the ith edge server is
Figure FDA0002907041940000021
type∈{cf，ig}；
The risk probability of the task data being attacked is
Figure FDA0002907041940000022
And its risk probability P (v)k) Probability of risk P that must be less than or equal to the scene settingmax；
S5, constructing a Markov decision process model in the mobile edge environment, which comprises the following steps:
s51, defining the state of the system
Figure FDA0002907041940000023
Wherein Wc(τ)＝{Wc，1(τ)，...，Wc，i(τ)，，...，Wc，n(τ) } represents a set of task nodes offloaded to the corresponding edge server;
Figure FDA0002907041940000024
representing a transmission rate between the mobile device and the corresponding edge server;
s52, defining the action of the system
Figure FDA0002907041940000025
Wherein a isc(τ)＝{ac，1(τ)，...，ac，i(τ)，，...，ac，n(τ) } indicates that the task nodes in the workflow are unloaded to a certain edge server;
Figure FDA0002907041940000026
indicating the security level of the cryptographic service selected by the mobile device when scheduling a task node in the workflow,
Figure FDA0002907041940000027
representing the security level of the guaranteed data integrity service selected by the mobile device when scheduling the task node in the workflow;
s53. defining reward R (tau) ═ T of systemend(vk) Wherein T isend(vk) Representing a node vkThe latest completion time of (c); latest time of completion
Figure FDA0002907041940000028
Figure FDA0002907041940000029
Comprises a task node vkStart time T ofstart(vk) Time of encryption
Figure FDA00029070419400000210
Time of flight
Figure FDA00029070419400000211
Waiting time
Figure FDA00029070419400000212
Decryption time
Figure FDA00029070419400000213
And execution time
Figure FDA00029070419400000214
S6, building a depth Q network:
the deep Q neural network comprises an estimation Q neural network, a target neural network and an experience pool;
the Q neural network and the target Q neural network have the same network structure, and the Q neural network can periodically transmit network parameters to the target neural network;
the experience pool is used for storing a state transition sample obtained by interaction with the environment in each time slice, and a fixed batch of quaternion groups are randomly extracted from the experience pool to train the estimation Q neural network in each learning;
s7, algorithm implementation:
a constant value epsilon is given as learning time, and all task nodes in the workflow are considered to be completed by one-time learning after learning and scheduling; in the workflow scheduling process, setting the current time slice as 0; at the beginning of the tau-th time slice, selecting and executing an action a (tau) by observing the current state s (tau) of the moving edge environment, calculating a reward R (s (tau), a (tau)) after executing the action and observing the state s (tau' +1) of the system after executing the action, and storing the reward R (s (tau), a (tau)) and the state s into an experience pool; when enough data is stored in the experience pool, sampling learning is started.
2. The method for workflow scheduling based on deep Q neural network in edge computing environment according to claim 1, wherein: setting the computing power of the mobile device in S1 to 10GHz/S, and then providing 5 edge servers with computing powers of 15GHz/S, 17.5GHz/S, 20GHz/S, 22.5GHz/S and 25GHz/S respectively.
3. The method for workflow scheduling based on deep Q neural network in edge computing environment according to claim 1, wherein: setting the number of the workflow nodes in the S2 as 100, setting the sum of the access degrees of other task nodes except the start node and the end node to be less than 5, and setting the probability of generating the dependency relationship between the two task nodes as 10%; the workload of each task node is subject to uniform distribution on 1-10 GHz.s, and the data size is subject to uniform distribution on 10-100 MB.
4. The method for workflow scheduling based on deep Q neural network in edge computing environment according to claim 1, wherein: s4, setting the risk coefficients of task data leakage in the process of transmitting task data to the edge server by the mobile device to be respectively
Figure FDA0002907041940000031
Figure FDA0002907041940000032
And
Figure FDA0002907041940000033
the risk coefficients of the modified task data are respectively set as
Figure FDA0002907041940000034
Figure FDA0002907041940000035
And
Figure FDA0002907041940000036
5. the method for workflow scheduling based on deep Q neural network in edge computing environment according to claim 1, wherein: start time in S53
Figure FDA0002907041940000037
pre(vk) Is v iskOf predecessor nodes, Tend(vh) Is any predecessor node vhThe latest time of completion of the process,
Figure FDA0002907041940000038
is to execute any predecessor nodeV ishV is sent to the edge serverhIs transmitted to the executing node v which is currently scheduledkThe time of the edge server of (1);
encryption time
Figure FDA0002907041940000039
Time of flight
Figure FDA00029070419400000310
Decryption time
Figure FDA00029070419400000311
Execution time
Figure FDA00029070419400000312
Waiting time
Figure FDA00029070419400000313
And the time for the ith edge server to finish executing all task nodes in the server queue.
CN202110074556.5A 2021-01-20 2021-01-20 Workflow scheduling method based on deep Q neural network in edge computing environment Withdrawn CN112905312A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110074556.5A CN112905312A (en) 2021-01-20 2021-01-20 Workflow scheduling method based on deep Q neural network in edge computing environment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110074556.5A CN112905312A (en) 2021-01-20 2021-01-20 Workflow scheduling method based on deep Q neural network in edge computing environment

Publications (1)

Publication Number Publication Date
CN112905312A true CN112905312A (en) 2021-06-04

Family

ID=76116522

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110074556.5A Withdrawn CN112905312A (en) 2021-01-20 2021-01-20 Workflow scheduling method based on deep Q neural network in edge computing environment

Country Status (1)

Country Link
CN (1) CN112905312A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113419830A (en) * 2021-06-23 2021-09-21 鹤壁国立光电科技股份有限公司 Multi-dimensional scheduling method and system based on neural network
CN113568675A (en) * 2021-07-08 2021-10-29 广东利通科技投资有限公司 Internet of vehicles edge calculation task unloading method based on layered reinforcement learning
CN115114030A (en) * 2022-07-20 2022-09-27 杭州电子科技大学 Online multi-workflow scheduling method based on reinforcement learning
CN113568675B (en) * 2021-07-08 2024-04-12 广东利通科技投资有限公司 Internet of vehicles edge computing task unloading method based on hierarchical reinforcement learning

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113419830A (en) * 2021-06-23 2021-09-21 鹤壁国立光电科技股份有限公司 Multi-dimensional scheduling method and system based on neural network
CN113419830B (en) * 2021-06-23 2023-02-03 鹤壁国立光电科技股份有限公司 Multi-dimensional scheduling method and system based on neural network
CN113568675A (en) * 2021-07-08 2021-10-29 广东利通科技投资有限公司 Internet of vehicles edge calculation task unloading method based on layered reinforcement learning
CN113568675B (en) * 2021-07-08 2024-04-12 广东利通科技投资有限公司 Internet of vehicles edge computing task unloading method based on hierarchical reinforcement learning
CN115114030A (en) * 2022-07-20 2022-09-27 杭州电子科技大学 Online multi-workflow scheduling method based on reinforcement learning
CN115114030B (en) * 2022-07-20 2023-06-16 杭州电子科技大学 On-line multi-workflow scheduling method based on reinforcement learning

Similar Documents

Publication Publication Date Title
Tang et al. Deep reinforcement learning for task offloading in mobile edge computing systems
Zhao et al. Offloading tasks with dependency and service caching in mobile edge computing
CN107911478B (en) Multi-user calculation unloading method and device based on chemical reaction optimization algorithm
Wang et al. Learning-based task offloading for delay-sensitive applications in dynamic fog networks
Liu et al. A reinforcement learning-based resource allocation scheme for cloud robotics
CN110096362B (en) Multitask unloading method based on edge server cooperation
CN113220356B (en) User computing task unloading method in mobile edge computing
Kiani et al. Hierarchical capacity provisioning for fog computing
Yuan et al. Online dispatching and fair scheduling of edge computing tasks: A learning-based approach
CN113064671A (en) Multi-agent-based edge cloud extensible task unloading method
CN111711962B (en) Cooperative scheduling method for subtasks of mobile edge computing system
CN112905312A (en) Workflow scheduling method based on deep Q neural network in edge computing environment
CN112988285B (en) Task unloading method and device, electronic equipment and storage medium
Tang et al. Dependent task offloading for multiple jobs in edge computing
KR20230007941A (en) Edge computational task offloading scheme using reinforcement learning for IIoT scenario
Liu et al. Fine-grained offloading for multi-access edge computing with actor-critic federated learning
Gao et al. Com-DDPG: A multiagent reinforcement learning-based offloading strategy for mobile edge computing
Lorido-Botran et al. ImpalaE: Towards an optimal policy for efficient resource management at the edge
Yang et al. Cooperative task offloading for mobile edge computing based on multi-agent deep reinforcement learning
Qiao et al. Task migration computation offloading with low delay for mobile edge computing in vehicular networks
CN115794323A (en) Task scheduling method, device, server and storage medium
CN113190342A (en) Method and system architecture for multi-application fine-grained unloading of cloud-edge cooperative network
Yu et al. A sum-of-ratios multi-dimensional-knapsack decomposition for DNN resource scheduling
Zhang et al. Multi-agent deep reinforcement learning for online request scheduling in edge cooperation networks
CN113176936A (en) QoE-aware distributed edge task scheduling and resource management method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WW01 Invention patent application withdrawn after publication
WW01 Invention patent application withdrawn after publication

Application publication date: 20210604